X · @teortaxesTex
· X / Twitter
I've been an advocate of <2% activation ratio for a looong time. Reminder! We have every reason to believe it's possible to keep gaining in the <1% re…
I've been an advocate of LatentMoE> 16 activated experts out of 896