arXiv 2603.191662026-03-19cs.ROcs.AIcs.CLcs.CVcs.LG

含义与测量：多智能体概率性视觉语言导航

Meanings and Measurements: Multi-Agent Probabilistic Grounding for Vision-Language Navigation

Arizona State University（亚利桑那州立大学）；Haverford College（哈弗福德学院）

Swagat Padhan, Lakshya Jain, Bhavya Minesh Shah, Omkar Patil, Thao Nguyen, Nakul Gopalan

备注：: Equal contribution: Swagat Padhan and Lakshya Jain, 9 pages, 6 figures, paper website: https://lakshya-asu.github.io/Meanings-Measurements-Multi-Agent-Probabilistic-Grounding/

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出MAPG框架，通过分解语言查询并利用VLM进行语义 grounding，解决复杂度量-语义语言查询的难题，同时引入MAPG-Bench评估指标，展示在现实机器人中的应用效果。

摘要：

协作的机器人必须将自然语言目标转换为可操作的物理决策。例如，执行“向冰箱右侧两米前进”的命令需要在3D场景中接地语义引用、空间关系和度量约束。尽管最近的视觉语言模型（VLMs）展示了强大的语义接地能力，但它们并未专门设计来推理物理定义空间中的度量约束。在本工作中，我们实证表明，最先进的VLM-based grounding方法在处理复杂度量-语义语言查询时存在困难。为了解决这一限制，我们提出了MAPG（多智能体概率性接地），一种智能体框架，将语言查询分解为结构化子组件，并查询VLM来接地每个组件。MAPG随后概率性地组合这些接地输出，以在3D空间中产生度量一致、可操作的决策。我们在HM-EQA基准上评估MAPG，并展示了相对于强基线的一致性能提升。此外，我们引入了一个新的基准MAPG-Bench，专门用于评估度量-语义目标接地，填补了现有语言接地评估中的空白。我们还展示了一个现实机器人演示，表明当有结构化场景表示时，MAPG在仿真之外也能转移。

英文摘要：

Robots collaborating with humans must convert natural language goals into actionable, physically grounded decisions. For example, executing a command such as "go two meters to the right of the fridge" requires grounding semantic references, spatial relations, and metric constraints within a 3D scene. While recent vision language models (VLMs) demonstrate strong semantic grounding capabilities, they are not explicitly designed to reason about metric constraints in physically defined spaces. In this work, we empirically demonstrate that state-of-the-art VLM-based grounding approaches struggle with complex metric-semantic language queries. To address this limitation, we propose MAPG (Multi-Agent Probabilistic Grounding), an agentic framework that decomposes language queries into structured subcomponents and queries a VLM to ground each component. MAPG then probabilistically composes these grounded outputs to produce metrically consistent, actionable decisions in 3D space. We evaluate MAPG on the HM-EQA benchmark and show consistent performance improvements over strong baselines. Furthermore, we introduce a new benchmark, MAPG-Bench, specifically designed to evaluate metric-semantic goal grounding, addressing a gap in existing language grounding evaluations. We also present a real-world robot demonstration showing that MAPG transfers beyond simulation when a structured scene representation is available.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。