ShortOPD: Recovering Pruned LLMs with Short-to-Long On-Policy Distillation
arXiv:2607.13124v1 Announce Type: new Abstract: Structured pruning is a hardware-friendly way to compress LLMs, but it is mostly validated on multiple-choice recognition tasks, while the same…