Update README.md

README.md

@@ -93,6 +93,25 @@ https://github.com/josephrocca/rwkv-v4-web
 
 For the old RWKV-2: see the release here for a 27M params model on enwik8 with 0.72 BPC(dev). Run run.py in https://github.com/BlinkDL/RWKV-LM/tree/main/RWKV-v2-RNN. You can even run it in your browser: https://github.com/BlinkDL/AI-Writer/tree/main/docs/eng https://blinkdl.github.io/AI-Writer/eng/ (this is using tf.js WASM single-thread mode).
 
+I have an idea to quantize a matrix with outliers:
+```python
+import numpy as np
+
+# the original M, with outliers
+M = np.array([[1,   2,   1,  2],[2,  100,    2, 10],[1,   2,   1, 2],[2,   1, 20, 1]])
+
+# the scaled M, without outliers
+Q = np.array([[1, 0.2, 0.1,  2],[0.4,  2, 0.04, 2], [1, 0.2, 0.1, 2],[2, 0.1,  2, 1]])
+# we can find optimal a & b to minimize inference error after quantization
+a = np.array([1, 10, 10, 1])
+b = np.array([1, 5, 1, 1])
+
+# test M.v with random v - the results will be the same
+v = np.array([1.23, 5.44, 9.75, 2.98])
+print(M.dot(v))
+print(Q.dot(v * a) * b)
+```
+
 ### Training / Fine-tuning
 
 **Training RWKV-4 from scratch:** run train.py, which by default is using the enwik8 dataset (unzip https://data.deepai.org/enwik8.zip).