From 883560d143f9ee391c8c101e1b021dfdf9a90e3d Mon Sep 17 00:00:00 2001 From: randaller Date: Sun, 5 Mar 2023 20:17:52 +0300 Subject: [PATCH] Update README.md --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 4c7ca6d..4380e35 100755 --- a/README.md +++ b/README.md @@ -69,7 +69,7 @@ Running model with single prompt on Windows computer equipped with 12700k, fast ### RAM usage optimization By default, torch uses Float32 precision while running on CPU, which leads, for example, to use 44 GB of RAM for 7B model. We may use Bfloat16 precision on CPU too, which decreases RAM consumption/2, down to 22 GB for 7B model, but inference processing much slower. -Uncomment this line in the example-cpu.py to enable Bfloat16 and save memory. +Uncomment this line in the example-cpu.py or example-chat.py to enable Bfloat16 and save memory. ``` torch.set_default_dtype(torch.bfloat16) ```