From 89c953b94165e6bc2ed9c716189746a0b0ae068a Mon Sep 17 00:00:00 2001 From: PENG Bo <33809201+BlinkDL@users.noreply.github.com> Date: Wed, 11 May 2022 22:56:52 +0800 Subject: [PATCH] Update README.md --- README.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/README.md b/README.md index 3bcb96f..40a1634 100644 --- a/README.md +++ b/README.md @@ -6,6 +6,8 @@ RWKV v2 is a RNN with Transformer-level performance, which can also be directly So it's combining the best of RNN and transformer - great performance, fast inference, saves VRAM, fast training, "infinite" ctx_len, and free sentence embedding. +Reddit discussion: https://www.reddit.com/r/MachineLearning/comments/umq908/r_rwkvv2rnn_a_parallelizable_rnn_with/ + I am training it on the Pile (https://github.com/BlinkDL/RWKV-v2-RNN-Pile) and it might reach GPT-Neo performance within 100B tokens: ![RWKV-v2-430M-Pile](RWKV-v2-430M-Pile.png)