Researchers have introduced HarnessVLN, a zero-shot, training-free framework designed to unify instruction-following and object-goal navigation by leveraging Multimodal Large Language Models (MLLMs). Rather than relying on task-specific training, the framework utilizes an "Agent Harness" to coordinate perception, memory, and execution tools through a single interface. This system enhances navigation reliability by validating planner proposals for geometric feasibility, evidential support, and subgoal consistency before execution. To maintain situational awareness, HarnessVLN manages a hierarchical event memory alongside a persistent Spatiotemporal Graph, which tracks task progress and contextualizes execution failures. Structured feedback from execution is fed back into this shared state to refine subsequent planning and recovery steps. Empirical evaluations demonstrate the framework's effectiveness across multiple benchmarks, achieving success rates of 60.8% on R2R, 53.9% on RxR, 76.0% on HM3D-v2, and 59.3% on HM3D-OVON. These results outperform existing training-free state-of-the-art methods. Furthermore, the authors validated the system's practical utility by successfully deploying the framework on a humanoid robot within real-world environments, proving its capacity for generalizable embodied navigation without extensive fine-tuning.
Read original
huggingface/daily-papers