OpenMap：通过开放词汇视觉-语言映射实现指令定位
OpenMap: Instruction Grounding via Open-Vocabulary Visual-Language Mapping
AI总结：
OpenMap提出零样本开放词汇视觉-语言地图，通过结构-语义一致性约束和LLM辅助指令定位模块，在ScanNet200和Matterport3D上实现优于基线的指令定位性能。
摘要：
将自然语言指令定位到视觉观察上是具身智能体在开放世界环境中运行的基础。视觉-语言映射的最新进展通过利用视觉-语言模型（VLMs）实现了可泛化的语义表示。然而，这些方法在将自由形式的语言命令与特定场景实例对齐方面往往表现不足，原因在于实例级语义一致性和指令解释两方面的局限性。我们提出了OpenMap，一种为零样本开放词汇视觉-语言地图，专为导航任务中的精确指令定位而设计。为解决跨视图的语义不一致问题，我们引入了一种结构-语义一致性约束，该约束联合考虑全局几何结构和视觉-语言相似性，以指导稳健的3D实例级聚合。为改进指令解释，我们提出了一种基于LLM辅助的指令到实例定位模块，该模块通过结合空间上下文和表达性目标描述实现细粒度实例选择。我们在ScanNet200和Matterport3D上评估了OpenMap，涵盖了语义映射和指令到目标检索任务。实验结果表明，OpenMap在零样本设置下优于最先进的基线方法，证明了我们的方法在弥合自由形式语言与3D感知之间差距以支持具身导航方面的有效性。
英文摘要：
Grounding natural language instructions to visual observations is fundamental for embodied agents operating in open-world environments. Recent advances in visual-language mapping have enabled generalizable semantic representations by leveraging vision-language models (VLMs). However, these methods often fall short in aligning free-form language commands with specific scene instances, due to limitations in both instance-level semantic consistency and instruction interpretation. We present OpenMap, a zero-shot open-vocabulary visual-language map designed for accurate instruction grounding in navigation tasks. To address semantic inconsistencies across views, we introduce a Structural-Semantic Consensus constraint that jointly considers global geometric structure and vision-language similarity to guide robust 3D instance-level aggregation. To improve instruction interpretation, we propose an LLM-assisted Instruction-to-Instance Grounding module that enables fine-grained instance selection by incorporating spatial context and expressive target descriptions. We evaluate OpenMap on ScanNet200 and Matterport3D, covering both semantic mapping and instruction-to-target retrieval tasks. Experimental results show that OpenMap outperforms state-of-the-art baselines in zero-shot settings, demonstrating the effectiveness of our method in bridging free-form language and 3D perception for embodied navigation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
