X · @teortaxesTex
· X / Twitter
RT サメQCU: https://x.com/tokenbender/status/2081776672152686883 "kimi stopped using positional embedding" inside the text: "kimi uses a linear attent…
RT サメQCUhttps://x.com/tokenbender/status/2081776672152686883"kimi stopped using positional embedding"inside the text:"kimi uses a linear attention layer which is a positional embedding for other subsequent layers"tokenbender: kimi guys noped the fuck out of rope at this scale.no positional encoding anywhere what so eve