From f11b97ae483302a37b04d5cb3bf7af4a13121b20 Mon Sep 17 00:00:00 2001 From: PENG Bo <33809201+BlinkDL@users.noreply.github.com> Date: Sat, 26 Mar 2022 23:26:06 +0800 Subject: [PATCH] Update README.md --- README.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/README.md b/README.md index a37dc4a..cbe1eb0 100644 --- a/README.md +++ b/README.md @@ -8,6 +8,8 @@ You only need x_t, a_t, b_t of position t to compute the vectors for position t+ Hence it can be 100x faster than GPT, and 100x more VRAM friendly. +See the release for a 27M params model on enwik8 with 1.65 dev perplexity (0.72 BPC). + ## How it works The a b c d factors work together to build a time-decay curve: X, 1, W, W^2, W^3, ...