X · @teortaxesTex
· X / Twitter
Zyphra fits a scaling law for plasticity loss in continuously trained LLMs. What can we do to push the point of rigidity onset towards infinity? I rec…
Zyphra fits a scaling law for plasticity loss in continuously trained LLMs. What can we do to push the point of rigidity onset towards infinity? I recall Sutton's team could only come up with continual backpropagation (random reinitialization of some units)… suboptimal.Zyphra: Zyphra is sharing our first work in contin