novarobot
/
llama-cpu
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Update README.md

Browse Source
main
randaller 3 years ago committed by GitHub
parent bf61952012
commit 478111c1f3
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 1 additions and 1 deletions
Show Stats Download Patch File Download Diff File

2
README.md
Unescape Escape View File

@ -68,7 +68,7 @@ Running model with single prompt on Windows computer equipped with 12700k, fast
| model | RAM usage, fp32 | RAM usage, bf16 | fp32 inference | bf16 inference | fp32 load model | | model | RAM usage, fp32 | RAM usage, bf16 | fp32 inference | bf16 inference | fp32 load model |
| ------------- | ------------- | ------------- | ------------- | ------------- | ------------- | | ------------- | ------------- | ------------- | ------------- | ------------- | ------------- |
| 7B | 44 Gb, peak 56 Gb | 22 Gb | 170 seconds | 850 seconds | 23 seconds | | 7B | 44 Gb, peak 56 Gb | 22 Gb | 170 seconds | 850 seconds | 23 seconds |
| 13B | 77 Gb, peak 100 Gb | 38 Gb | 340 seconds | | 61 seconds | | 13B | 77 Gb, peak 100 Gb | 38 Gb | 340 seconds | 38 minutes | 61 seconds |
| 30B | 180 Gb, peak 258 Gb | 89 Gb | 48 minutes | 67 minutes | 372 seconds | | 30B | 180 Gb, peak 258 Gb | 89 Gb | 48 minutes | 67 minutes | 372 seconds |
### RAM usage optimization ### RAM usage optimization

Write Preview
Loading…
Cancel
Save