RT tianhang zhuRe @ChujieZheng i was responsible for delivery of the RL ckpt. so i think head of rl is accurate description for people to quickly…
The thing that impresses me here is not that it works (duh of course it's algorithms), but that they suffer a perplexity hit without commensurate downstream…
RT Francesco BertolottiQuite a jump from vanilla OPD. Here the authors use the difference between a privileged teacher and privileged student to compute a token-level advantage.…
I wonder how much economic value DeepSeek has just created with DSpark. Imagine if *every* Chinese provider improves throughput by 50% 3 months faster. Or improves…
> Devin Fusion builds on this research while avoiding common pitfalls of model routing, like expensive cache misses and lack of generalization.Cognition finally showing serious engineeringCognition:…
That's what you do when you're confident that no particular technology or trick is your moat. Other labs are welcome to also copy CSA+HCA+all other nonsense…
It offends me that this is a useful heuristic35/1.6, 33/1.6, 30/1, 28/0.75, 32/0.28The higher the ratio of tokens to total parameters, the more impressive the model…
let the cat wars beginelie: le chaton long > le chaton fat for now
One needs the chutzpah to pose a question like "why not drop matmul". But I'm afraid we're not getting many answers.ali: After reading up a bit…
/r/LocalLLaMADario is the #1 designated villain for local guysI recall when they didn't know him and hated Sam…total Chinese soft power victory on the open AI…