From 760db55fa6896f682a40fc7b66df974c894acbc0 Mon Sep 17 00:00:00 2001 From: PENG Bo <33809201+BlinkDL@users.noreply.github.com> Date: Sun, 26 Feb 2023 05:49:06 +0800 Subject: [PATCH] Update README.md --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 4704415..2eb7ae7 100644 --- a/README.md +++ b/README.md @@ -192,7 +192,7 @@ out.write(ss + "\n") ### Some ideas -1. Now time decay is like 0.999^T (0.999 is learnable). Change it to something like (0.999^T + 0.1) where 0.1 is learnable too. The 0.1 part will be kept forever. Or try A^T + B^T = fast-decay + slow-decay. +1. Now time decay is like 0.999^T (0.999 is learnable). Change it to something like (0.999^T + 0.1) where 0.1 is learnable too. The 0.1 part will be kept forever. Or, A^T + B^T + C = fast-decay + slow-decay + constant. Can even use different formulas (for example, x^2 instead of e^x, or, without normalization). 2. Use complex-valued decay (so, rotation instead of decay) in some channels.