LessWrong AI
· Communities
CLT Features Sharpen the Cyclical Day-of-Week Manifold in Gemma-2-2b
tl;dr: I reproduced the Goodfire lab's cyclical manifold result using days-of-the-week on Gemma-2-2b and found that Anthropic's pre-trained CLT features produce an even cleaner cyclical manifold with more distinct features than the raw activations or output probabilities. Colab notebook implementation can be found here