SAS proposes a post‑training attention sparsification technique that selects a small set of context units per query, thereby lowering the quadratic attention cost of pretrained Transformers. Unlike prior trainable selectors that rely on hard Top‑K selection—which blocks gradients and necessitates distillation of dense attention—SAS optimizes the selector end‑to‑end, allowing gradients to flow directly from the language‑modeling loss. This yields a selector that ranks context units effectively without the need for intermediate distillation.