Untied Ulysses: Memory-Efficient Context Parallelism via Headwise Chunking
arXiv:2602.21196v2 Announce Type: replace Abstract: Efficiently processing long sequences with Transformer models usually requires splitting the computations across accelerators via context parallelism. The dominant approaches in…