In the ECCV 2026 Wearable AI Challenge, the "Ambient @ EgoProactive" submission achieved first place in the large-model division and second place in the <=2B parameter division by developing a proactive egocentric assistance system. The core objective was to determine, following every eight-second segment of egocentric video, whether a wearable assistant should intervene or remain silent. The researchers implemented two primary technical innovations. First, they transitioned from free-form text generation to a single-token classification method for intervention timing. By predicting "yes" or "no" and calculating decisions via renormalized probabilities rather than generating sequences like "interrupt <utterance>", the model achieved significant performance gains, specifically increasing the macro-F1 score by 0.249 and the G-mean by 0.30. Second, to address the scarcity of labeled data, the team utilized a tool-calling video agent to generate visual-grounded supervision by inspecting clips and assigning intervention timestamps. This approach proved superior to a narration-only alternative, which was four times larger and ten times more cost-effective but exhibited poorer transferability. The results indicate that visual grounding is a more critical factor for model performance in egocentric assistance tasks than the sheer volume of annotation data.

Read original