X · @teortaxesTex
· X / Twitter
> done right cheeky but this is exciting. Tencent is still using GQA in their best model, which I just criticized them for. Using landmark tokens is c…
> done rightcheekybut this is exciting. Tencent is still using GQA in their best model, which I just criticized them for. Using landmark tokens is clever.Rosinality: https://arxiv.org/abs/2607.02980Sparse Attention. Each chunk is scored by aggregated scores using landmark tokens. (And these aggregated scores are traine