r/MachineLearning June 21, 2026 · Communities

An Update on Matrix Recurrent Units, an Attention Alternative [R]

I recently revisited my matrix recurrent units algorithm (the MRU), a novel linear-time sequence architecture I created as an alternative to attention. I explain it in depth at the repo, but the gist is the MRU works by transforming the embedding into an input state matrix, cumulatively multiplying the matrices across