novarobot
/
RWKV-LM
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Update README.md

Browse Source
main
PENG Bo 3 years ago committed by GitHub
parent 4c946fac2e
commit c22c7dadca
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 2 additions and 0 deletions
Show Stats Download Patch File Download Diff File

2
README.md
Unescape Escape View File

@ -20,6 +20,8 @@ I am training RWKV-4 3B and 7B on the Pile (https://wandb.ai/blinkdl/RWKV-v4-Pil
![RWKV-v4-1.5B-Pile](RWKV-v4-1.5B-Pile.png) ![RWKV-v4-1.5B-Pile](RWKV-v4-1.5B-Pile.png)
![RWKV-eval](RWKV-eval.png)
All of the trained models will be open-source. Inference is very fast (only matrix-vector multiplications, no matrix-matrix multiplications) even on CPUs, so you can even run a LLM on your phone. All of the trained models will be open-source. Inference is very fast (only matrix-vector multiplications, no matrix-matrix multiplications) even on CPUs, so you can even run a LLM on your phone.
How it works: RWKV gathers information to a number of channels, which are also decaying with different speeds as you move to the next token. It's very simple once you understand it. How it works: RWKV gathers information to a number of channels, which are also decaying with different speeds as you move to the next token. It's very simple once you understand it.

Write Preview
Loading…
Cancel
Save