Update README.md

main
PENG Bo 4 years ago committed by GitHub
parent e541d93b97
commit 2951b2895a
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23

@ -2,7 +2,7 @@
## RWKV v2: RNN with Transformer Performance ## RWKV v2: RNN with Transformer Performance
RWKV v2 is a RNN which can also be directly trained like a GPT transformer. You only need x_t, a_t, b_t of position t to compute the vectors for position t+1. Hence it can be 100x faster than GPT, and 100x more VRAM friendly. RWKV v2 is a RNN which can also be directly trained like a GPT transformer. You only need x_t, a_t, b_t of position t to compute the vectors for position t+1. Hence it can be 100x faster than GPT, and 100x more VRAM friendly, and you get a free sentence embedding.
See the release for a **27M params model on enwik8 with 0.72 BPC(dev)**. See the release for a **27M params model on enwik8 with 0.72 BPC(dev)**.

Loading…
Cancel
Save