LessWrong AI
· Communities
An Induction Head in Disguise: Chasing Grammar in a Character-Level Transformer
Confident in the measurements — copying score, OV logits, attention patterns. More exploratory on the interpretations of what the head "knows," two of which I ended up disconfirming myself. My first mechanistic interpretability post; corrections welcome.Full analysis and code: github/Ameya-bit/QuotesByNicheI trained a