arXiv cs.LG
· Papers
MultiHashFormer: Hash-based Generative Language Models
arXiv:2606.28057v1 Announce Type: cross Abstract: Language models (LMs) represent tokens using embedding matrices that scale linearly with the vocabulary size. To constrain the parameter footprint, prior work proposes hashing many tokens into a single vector within encoder-only models. While this offers parameter effic