X · @teortaxesTex
· X / Twitter
The issue isn't so much about logits, this is a red herring imo the main problem is on-policy vs off-policy RL. Chinese labs are shifting towards MOPD…
The issue isn't so much about logits, this is a red herring imothe main problem is on-policy vs off-policy RL. Chinese labs are shifting towards MOPD. This is way more important for sustained progress than Claude's trajectorieswill depue: it’s easy to justify why distilling without logits should only be a small constan