Researchers have introduced OmniScope, a training-free token compression framework designed to optimize inference for omnimodal large language models (OmniLLMs) by addressing cross-modal salience mismatch. Unlike existing methods that rely on one modality to guide the compression of another, OmniScope treats the user query as a shared semantic anchor while calculating relevance scores independently for audio and video. This modality-decoupled approach prevents the accidental removal of critical cues when temporal relevance peaks differ between modalities. For visual data, the framework employs an anchor-delta strategy to prune tokens while preserving global context and temporal fluctuations. For audio, it merges tokens within each one-second interval to minimize redundancy while maintaining continuity. Evaluated on two Qwen2.5-Omni model scales across four audio-video benchmarks, OmniScope consistently achieved superior average accuracy compared to existing methods. At a 25% token retention rate, the framework demonstrated significant efficiency gains, providing up to a 3.53x prefill speedup and reducing GPU memory consumption by more than 15%. Notably, these performance improvements were achieved with a minimal accuracy penalty of only 0.35 points. The study concludes that decoupling salience estimation from the shared query is an effective design principle for efficient OmniLLM inference.
Read original
huggingface/daily-papers