The study challenges the prevailing view that input length degrades language model retrieval performance, demonstrating instead that competitive distractors drive "context rot." Evaluating five open models (0.5B–3B parameters across four families), researchers found neutral filler tokens up to 32K tokens preserve accuracy at 0.97–1.00 on Qwen2.5-1.5B. However, inserting statements formatted as evidence with rival values for the queried key reduces accuracy to 0.16–0.33; a single rival multiplies loss by 9–110×, with each additional rival compounding degradation. The mechanism is attentional competition: the attention mass retained on the true evidence explains up to 58% of loss variance across 126,473 trials, while log length explains ≤8%. The masked-filler protocol, which previously suggested length alone hurts performance (reproducing 11–18 point MMLU drops at 30K tokens), fails a visible control in retrieval: masking filler costs 45 accuracy points at 32K tokens, whereas visible filler costs none. The authors conclude retrieval failure stems from attention competition, not length, and release their probe and trial-level measurements.

Read original