arXiv cs.CL
· Papers
Cross-Tokenizer On-Policy Distillation via Byte-Prefix Marginalization
arXiv:2607.22334v1 Announce Type: cross Abstract: Open-weight language models from different families exhibit complementary capabilities, motivating their consolidation into a compact student through on-policy distillation (OPD). However, full-vocabulary OPD typically assumes a shared tokenizer, while existing cross-to