arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2502.13451cs.RO
收藏

MapNav: 一种通过标注语义地图的新型记忆表示方法用于视觉-语言导航

MapNav: A Novel Memory Representation via Annotated Semantic Maps for Vision-and-Language Navigation

The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));Beijing Academy of Artificial Intelligence(北京人工智能研究院);Tsinghua University(清华大学);Beijing Innovation Center of Humanoid Robotics Co., Ltd.(北京人形机器人创新中心有限公司);School of Computer Science, Wuhan University(武汉大学计算机学院);State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University(北京大学多媒体信息处理国家重点实验室,计算机学院)

Lingfeng Zhang, Xiaoshuai Hao, Qinwen Xu, Qiang Zhang, Xinyao Zhang, Pengwei Wang, Jing Zhang, Zhongyuan Wang, Shanghang Zhang, Renjing Xu

AI总结:

本文提出MapNav,通过标注语义地图替代传统历史帧,提升视觉-语言导航任务的性能,实验表明其在模拟和现实环境中均达到SOTA水平。

摘要:

视觉-语言导航(VLN)是具身AI中的关键任务,要求智能体在多样且未见过的环境中导航时遵循自然语言指令。传统方法依赖历史观测作为时空上下文进行决策,导致显著的存储和计算开销。本文介绍MapNav,一种新型端到端的VLN模型,利用标注语义地图(ASM)替代历史帧。具体而言,我们的方法在每个回合开始时构建一个自上而下的语义地图,并在每个时间步更新它,允许精确的对象映射和结构化的导航信息。然后,我们通过显式的文本标签增强此地图,将抽象语义转化为清晰的导航提示,并生成我们的ASM。MapNav智能体使用构建的ASM作为输入,并利用VLM的强大端到端能力来增强VLN。大量实验表明,MapNav在模拟和现实环境中均达到SOTA水平,验证了我们方法的有效性。此外,我们还将发布我们的ASM生成源代码和数据集以确保可重复性,为该领域贡献了有价值的资源。我们相信,所提出的MapNav可以作为VLN中的新记忆表示方法,为该领域的未来研究铺平道路。

英文摘要:

Vision-and-language navigation (VLN) is a key task in Embodied AI, requiring agents to navigate diverse and unseen environments while following natural language instructions. Traditional approaches rely heavily on historical observations as spatio-temporal contexts for decision making, leading to significant storage and computational overhead. In this paper, we introduce MapNav, a novel end-to-end VLN model that leverages Annotated Semantic Map (ASM) to replace historical frames. Specifically, our approach constructs a top-down semantic map at the start of each episode and update it at each timestep, allowing for precise object mapping and structured navigation information. Then, we enhance this map with explicit textual labels for key regions, transforming abstract semantics into clear navigation cues and generate our ASM. MapNav agent using the constructed ASM as input, and use the powerful end-to-end capabilities of VLM to empower VLN. Extensive experiments demonstrate that MapNav achieves state-of-the-art (SOTA) performance in both simulated and real-world environments, validating the effectiveness of our method. Moreover, we will release our ASM generation source code and dataset to ensure reproducibility, contributing valuable resources to the field. We believe that our proposed MapNav can be used as a new memory representation method in VLN, paving the way for future research in this field.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑