MiniMax Sparse Attention (MSA)
MiniMax Sparse Attention (MSA): Scaling Long-Context Capabilities via Blockwise Sparsity MiniMax Sparse Attention (MSA) introduces a blockwise sparse attention mechanism built upon Grouped Query Attention (GQA) to mitiga…
→ View original source