arXiv cs.CL
· Papers
Geometry-Guided Layerwise FFN Width Allocation in Transformers
arXiv:2608.02064v1 Announce Type: cross Abstract: Feed-forward networks (FFNs) account for a large fraction of Transformer parameters, yet their hidden width is usually constant across depth. We ask whether this capacity can instead be allocated from a forward-pass measurement of layer behavior. We view each FFN as tra