X · @teortaxesTex
· X / Twitter
…would be devastating if it turned out that sparsity just doesn't scale and Ant brute forced to True Science of Pretraining on chonkers. (almost inco…
…would be devastating if it turned out that sparsity just doesn't scale and Ant brute forced to True Science of Pretraining on chonkers.(almost inconceivable on priors, but there are enough contingent choices in conventional MoE design that it could be true. Router, damn you…)will brown: love @dylan522p podcasts bc he