Researchers have introduced ReToken, a lightweight method designed to enhance the performance of vision-language models (VLMs) when navigating long visual contexts. To address the computational burden of processing excessive tokens and the performance degradation caused by distractors, ReToken utilizes a single learnable embedding that acts as an explicit retrieval target. This mechanism enables the model to select a sparse subset of query-relevant visual tokens directly from a pre-filled KV cache. Notably, the method requires only a small image-QA dataset for training and is efficient enough to perform both training and long-video inference on a single H100 GPU. Empirical results demonstrate significant improvements across multiple benchmarks: on the Visual Haystacks dataset, ReToken boosted the performance of Qwen3VL-8B by 13.4 points and InternVL3.5 by 12.4 points, representing a relative improvement of over 20%. Furthermore, the approach facilitates zero-shot transfer to long video tasks on LVBench, where Qwen3VL-8B achieved an 8.0-point gain. By focusing on sparse token selection, ReToken effectively mitigates the memory constraints and noise issues inherent in large-scale visual retrieval tasks.

Read original