AgenticNav：零样本视觉与语言导航作为工具调用框架
AgenticNav: Zero-Shot Vision-and-Language Navigation as a Tool-Calling Harness
AI总结：
提出AgenticNav，通过将动作、深度和记忆作为可调用工具暴露给VLM，实现零样本连续环境导航，在R2R-CE基准上达到SOTA性能。
摘要：
连续环境中的零样本视觉与语言导航（VLN-CE）最近随着大型视觉语言模型（VLM）的出现而变得可行。然而，现有方法通常依赖学习到的航点预测器来提出可导航动作，这严重限制了模型的动作空间，并且未能有效利用深度输入。此外，记忆通常通过累积包含大量无关上下文的冗长文本或视觉历史，或通过检索跨回合经验来处理，这削弱了零样本设置。在本文中，我们将零样本VLN-CE重新思考为VLM与环境之间的代理接口，并提出了AgenticNav，这是一个轻量级导航框架，将动作、深度和记忆暴露为可调用的工具。动作工具允许VLM直接选择RGB观测中的目标像素，并将其转换为可执行运动，而不是从预测的航点中选择。深度通过按需像素深度工具暴露，使VLM能够在需要的地方请求精确的度量距离。对于记忆，AgenticNav提供了一个紧凑的地图图像，总结历史轨迹，并配有一个召回工具，允许VLM有选择地重新访问过去的视觉观测，而不会使提示上下文过载。在R2R-CE基准上，AgenticNav在相同VLM骨干下，在零样本方法中建立了新的最先进（SOTA）性能。真实世界验证进一步突显了其相比先前方法的零样本泛化能力。消融实验表明，我们的动作工具设计优于传统航点预测器，并且深度工具和代理记忆进一步促进了导航性能。
英文摘要：
Zero-shot vision-and-language navigation in continuous environments (VLN-CE) has recently become feasible with large vision-language models (VLMs). Existing methods typically rely on learned waypoint predictors to propose navigable actions. This limits the model's action space and fails to leverage depth inputs effectively. Moreover, memory is commonly handled by accumulating long textual or visual histories, which overwhelms the prompt context. In this paper, we rethink zero-shot VLN-CE as an agentic interface between the VLM and the environment, and present AgenticNav, a lightweight navigation harness that exposes action, depth, and memory as callable tools. Instead of choosing from predicted waypoints, the action tool allows the VLM to directly select a target pixel in RGB observations, converting it into executable motion. Depth is exposed through an on-demand pixel-depth tool, enabling the VLM to request precise metric distances only where they matter. For memory, AgenticNav uses an agentic memory architecture combining reasoning text history and a compact map image, paired with a recall tool that allows the VLM to selectively revisit past visual observations. On R2R-CE, AgenticNav achieves state-of-the-art (SOTA) zero-shot performance under same VLM comparison, reaching 76% success rate and 66.50% SPL. Ablation experiments validate the effectiveness of our tool and memory designs, while backbone evaluations demonstrate more consistent navigation gains as VLMs improve, highlighting the harness's potential to benefit from future VLM advances. Real-world experiments on two distinct robotic platforms further demonstrate its zero-shot generalization and adaptability.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
