arXiv cs.LG July 3, 2026 · Papers

$mu$pscaling small models: Principled warm starts and hyperparameter transfer

arXiv:2602.10545v2 Announce Type: replace Abstract: Modern large-scale neural networks are often trained and released in multiple sizes to accommodate diverse inference budgets. To improve efficiency, recent work has explored model upscaling: initializing larger models from trained smaller ones to accelerate convergenc