diff --git a/README.md b/README.md index 765f78f..50b03be 100644 --- a/README.md +++ b/README.md @@ -8,7 +8,9 @@ You only need x_t, a_t, b_t of position t to compute the vectors for position t+ Hence it can be 100x faster than GPT, and 100x more VRAM friendly. -I AM STILL TRAINING A LM TO TEST ITS CONVERGENCE. +The a b c d factors work together to build a time-decay curve: u, 1, w, w^2, w^3, ... + +Write out the formulas for "token at pos 2" and "token at pos 3" and you will get the idea. The model: