From 9fc5f22f5ef6ded2203cebfbf07d4c3501e39589 Mon Sep 17 00:00:00 2001
From: randaller <randaller@users.noreply.github.com>
Date: Sun, 19 Mar 2023 13:30:02 +0300
Subject: [PATCH] Update README.md

---
 README.md | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/README.md b/README.md
index d0b3f07..ee690dc 100644
--- a/README.md
+++ b/README.md
@@ -168,6 +168,16 @@ Trained model will be saved into [./trained] folder. Now you may launch inferenc
 python hf-inference-example.py
 ```
 
+### Bfloat16 optimization
+
+To save memory you may enable Bfloat16 processing.
+
+```
+# to save memory use bfloat16 on cpu
+import torch
+torch.set_default_dtype(torch.bfloat16)
+```
+
 ## Reference
 
 LLaMA: Open and Efficient Foundation Language Models -- https://arxiv.org/abs/2302.13971