LessWrong AI
· Communities
Small Models Have A Global Workspace
tl;dr: whether a model has one tracks modern pretraining, not parameter count, and not instruction-tuning.Here's the intuition almost everyone has about a "global workspace" in a language model. It's a big-model thing. Something that only switches on once you have tens of billions of parameters, or something the instru