novarobot
/
llama-cpu
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Update README.md

Browse Source
main
randaller 3 years ago committed by GitHub
parent 8030dbb616
commit 1fc88c863c
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 8 additions and 7 deletions
Show Stats Download Patch File Download Diff File

15
README.md
Unescape Escape View File

@ -17,6 +17,14 @@ pip install -e .
Once your request is approved, you will receive links to download the tokenizer and model files. Once your request is approved, you will receive links to download the tokenizer and model files.
Edit the `download.sh` script with the signed url provided in the email to download the model weights and tokenizer. Edit the `download.sh` script with the signed url provided in the email to download the model weights and tokenizer.
### CPU Inference
Place tokenizer.model and tokenizer_checklist.chk into /tokenizer folder
Place three files of 7B model into /model folder
Run it:
```
python example-cpu.py
```
### Inference ### Inference
The provided `example.py` can be run on a single or multi-gpu node with `torchrun` and will output completions for two pre-defined prompts. Using `TARGET_FOLDER` as defined in `download.sh`: The provided `example.py` can be run on a single or multi-gpu node with `torchrun` and will output completions for two pre-defined prompts. Using `TARGET_FOLDER` as defined in `download.sh`:
``` ```
@ -43,10 +51,3 @@ See [MODEL_CARD.md](MODEL_CARD.md)
### License ### License
See the [LICENSE](LICENSE) file. See the [LICENSE](LICENSE) file.
### CPU Inference
Place tokenizer.model and tokenizer_checklist.chk into /tokenizer folder
Place three files of 7B model into /model folder
Run python example-cpu.py

Write Preview
Loading…
Cancel
Save