X · @teortaxesTex
· X / Twitter
It offends me that this is a useful heuristic 35/1.6, 33/1.6, 30/1, 28/0.75, 32/0.28 The higher the ratio of tokens to total parameters, the more impr…
It offends me that this is a useful heuristic35/1.6, 33/1.6, 30/1, 28/0.75, 32/0.28The higher the ratio of tokens to total parameters, the more impressive the model turns out to be. (what's up with Minimax M3 tho, how many tokens). More Dakka winningρ:ɡeon: Pretrained on 35T tokens, which is definitely a decent amount,