From 5713df51ec9423b12e682de3ef989e8f883ae38e Mon Sep 17 00:00:00 2001 From: PENG Bo <33809201+BlinkDL@users.noreply.github.com> Date: Tue, 21 Mar 2023 02:57:18 +0800 Subject: [PATCH] Update README.md --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 8779611..2fe5597 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,6 @@ # The RWKV Language Model (and my LM tricks) -## RWKV: Parallelizable RNN with Transformer-level LLM Performance (pronounced as "RwaKuv", comes from 4 major params: R W K V) +## RWKV: Parallelizable RNN with Transformer-level LLM Performance (pronounced as "RwaKuv", from 4 major params: R W K V) RWKV is an RNN with Transformer-level LLM performance, which can also be directly trained like a GPT transformer (parallelizable). And it's 100% attention-free. You only need the hidden state at position t to compute the state at position t+1. You can use the "GPT" mode to quickly compute the hidden state for the "RNN" mode.