X · @teortaxesTex
· X / Twitter
RT Will Bui: Now that Attention Residuals is proven at scale with Kimi K3, here are the kernels.
RT Will BuiNow that Attention Residuals is proven at scale with Kimi K3, here are the kernels.Will Bui: 27x faster Attention Residuals!!! 🚀We implemented Block AttnRes as a pip-installable package.!pip install flash-attn-resNo annoying kernel nonsense.No compile/autograd plumbing.Call it like a regular PyTorch op.It ju