LessWrong AI
· Communities
Compressed Computation under L⁴ Loss is likely Computation in Superposition
SummaryNeural networks are widely assumed to use superposition to represent more features than they have dimensions. A stronger claim is that they also compute in superposition (CiS), i.e., implement more nonlinear functions than they have neurons (Hänni et al. 2024). CiS remains poorly understood, and until recently t