LessWrong AI
· Communities
Probing the loss-band sparsity assumption in Scientist AI
Epistemic status: ~1 hour of compute on a T4. Note that I just tried with one model, and one subspace. The volume finding seems solid; the curvature finding is suggestive and I checked whether it generalises (it doesn't clearly). I think the methodology is the main interesting idea, and the specific numbers are a start