NavBench：探究多模态大语言模型在具身导航中的能力
NavBench: Probing Multimodal Large Language Models for Embodied Navigation
AI总结：
NavBench 提出一个评估多模态大语言模型零样本具身导航能力的基准，涵盖导航理解与逐步执行，并发现地图上下文可提升决策准确性，而时间进度估计仍是主要挑战。
摘要：
多模态大语言模型（MLLMs）在视觉-语言任务中展现出强大的泛化能力，但它们在具身环境中理解和行动的能力仍未得到充分探索。我们提出了 NavBench，一个用于在零样本设置下评估 MLLMs 具身导航能力的基准。NavBench 包含两个组成部分：（1）导航理解，通过三个基于认知的任务进行评估，包括全局指令对齐、时间进度估计和局部观察-动作推理，涵盖 3,200 个问答对；（2）在 72 个室内场景中的 432 个回合中逐步执行，并按空间、认知和执行复杂度进行分层。为支持真实世界部署，我们引入了一条将 MLLMs 的输出转换为机器人动作的流水线。我们评估了闭源和开源模型，发现 GPT-4o 在各任务中表现良好，而较轻量的开源模型在较简单的情况下能够成功。结果还表明，理解得分较高的模型往往能获得更好的执行性能。提供基于地图的上下文可提高决策准确性，尤其是在中等难度场景中。然而，大多数模型在时间理解方面存在困难，特别是在导航过程中估计进度时，这可能构成一个关键挑战。
英文摘要：
Multimodal Large Language Models (MLLMs) have demonstrated strong generalization in vision-language tasks, yet their ability to understand and act within embodied environments remains underexplored. We present NavBench, a benchmark to evaluate the embodied navigation capabilities of MLLMs under zero-shot settings. NavBench consists of two components: (1) navigation comprehension, assessed through three cognitively grounded tasks including global instruction alignment, temporal progress estimation, and local observation-action reasoning, covering 3,200 question-answer pairs; and (2) step-by-step execution in 432 episodes across 72 indoor scenes, stratified by spatial, cognitive, and execution complexity. To support real-world deployment, we introduce a pipeline that converts MLLMs' outputs into robotic actions. We evaluate both proprietary and open-source models, finding that GPT-4o performs well across tasks, while lighter open-source models succeed in simpler cases. Results also show that models with higher comprehension scores tend to achieve better execution performance. Providing map-based context improves decision accuracy, especially in medium-difficulty scenarios. However, most models struggle with temporal understanding, particularly in estimating progress during navigation, which may pose a key challenge.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
