X · @teortaxesTex
· X / Twitter
One needs the chutzpah to pose a question like "why not drop matmul". But I'm afraid we're not getting many answers.
One needs the chutzpah to pose a question like "why not drop matmul". But I'm afraid we're not getting many answers.ali: After reading up a bit on ML research post transformer era, I was upset that it seems to have converged on hyper-optimizing matmul-based algorithms:(MHA, MQA, MLA, SWA, DSA, GQA, SWA-GQA, ABCDA [only