Update README.md

main
PENG Bo 3 years ago committed by GitHub
parent 14d21f5a00
commit 5713df51ec
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23

@ -1,6 +1,6 @@
# The RWKV Language Model (and my LM tricks) # The RWKV Language Model (and my LM tricks)
## RWKV: Parallelizable RNN with Transformer-level LLM Performance (pronounced as "RwaKuv", comes from 4 major params: R W K V) ## RWKV: Parallelizable RNN with Transformer-level LLM Performance (pronounced as "RwaKuv", from 4 major params: R W K V)
RWKV is an RNN with Transformer-level LLM performance, which can also be directly trained like a GPT transformer (parallelizable). And it's 100% attention-free. You only need the hidden state at position t to compute the state at position t+1. You can use the "GPT" mode to quickly compute the hidden state for the "RNN" mode. RWKV is an RNN with Transformer-level LLM performance, which can also be directly trained like a GPT transformer (parallelizable). And it's 100% attention-free. You only need the hidden state at position t to compute the state at position t+1. You can use the "GPT" mode to quickly compute the hidden state for the "RNN" mode.

Loading…
Cancel
Save