From f20564a15d18a7c8a1c31fa86aa9589fcb4e8e3d Mon Sep 17 00:00:00 2001 From: PENG Bo <33809201+BlinkDL@users.noreply.github.com> Date: Tue, 24 May 2022 16:07:38 +0800 Subject: [PATCH] Update README.md --- README.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/README.md b/README.md index 08c7bed..e8d4add 100644 --- a/README.md +++ b/README.md @@ -15,6 +15,8 @@ Tweet from Sepp Hochreiter (thank you!): https://twitter.com/HochreiterSepp/stat User feedback: > *I've so far toyed around the character-based model on our relatively small pre-training dataset (around 10GB of text), and the results are extremely good - similar ppl to models taking much, much longer to train.* +> *dear god rwkv is fast. i switched to another tab after starting training it from scratch & when i returned it was emitting plausible english & maori words, i left to go microwave some coffee & when i came back it was producing fully grammatically correct sentences.* + I am training a L24-D1024 RWKV-2 on the Pile (https://github.com/BlinkDL/RWKV-v2-RNN-Pile): ![RWKV-v2-430M-Pile](RWKV-v2-430M-Pile.png)