r/LocalLLaMA July 10, 2026 · Communities

tencent/HiLS-Attention-7B · Hugging Face

HiLS-Attention is a chunk-wise sparse attention mechanism that learns chunk selection end-to-end under the language-modeling loss, enabling native sparse training for efficient long-context modeling. This repository hosts the 7B checkpoint continued-trained on top of an OLMo3-style backbone. Model introduced in the pap