MapNav: A Novel Memory Representation via Annotated Semantic Maps for Vision-and-Language Navigation
The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));Beijing Academy of Artificial Intelligence(北京人工智能研究院);Tsinghua University(清华大学);Beijing Innovation Center of Humanoid Robotics Co., Ltd.(北京人形机器人创新中心有限公司);School of Computer Science, Wuhan University(武汉大学计算机学院);State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University(北京大学多媒体信息处理国家重点实验室,计算机学院)
Vision-and-language navigation (VLN) is a key task in Embodied AI, requiring agents to navigate diverse and unseen environments while following natural language instructions. Traditional approaches rely heavily on historical observations as spatio-temporal contexts for decision making, leading to significant storage and computational overhead. In this paper, we introduce MapNav, a novel end-to-end VLN model that leverages Annotated Semantic Map (ASM) to replace historical frames. Specifically, our approach constructs a top-down semantic map at the start of each episode and update it at each timestep, allowing for precise object mapping and structured navigation information. Then, we enhance this map with explicit textual labels for key regions, transforming abstract semantics into clear navigation cues and generate our ASM. MapNav agent using the constructed ASM as input, and use the powerful end-to-end capabilities of VLM to empower VLN. Extensive experiments demonstrate that MapNav achieves state-of-the-art (SOTA) performance in both simulated and real-world environments, validating the effectiveness of our method. Moreover, we will release our ASM generation source code and dataset to ensure reproducibility, contributing valuable resources to the field. We believe that our proposed MapNav can be used as a new memory representation method in VLN, paving the way for future research in this field.
GeoNav: Empowering MLLMs with dual-scale geospatial reasoning for language-goal aerial navigation
College of Systems Engineering, National University of Defense Technology(系统工程学院,国防科技大学);State Key Laboratory of Digital Intelligent Modeling and Simulation(数字智能建模与仿真国家重点实验室);BNRist, Tsinghua University(清华大学BNRist)
FloorPlan-VLN: A New Paradigm for Floor Plan Guided Vision-Language Navigation
New Laboratory of Pattern Recognition (NLPR), Institute of Automation, Chinese Academy of Sciences and School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院自动化研究所新型模式识别实验室,中国科学院人工智能学院);AMap, Alibaba Group(阿里巴巴集团高德地图);BAAI(北京人工智能研究院);Xiaomi Robotics Lab(小米机器人实验室);FiveAges
Aux-Think: Exploring Reasoning Strategies for Data-Efficient Vision-Language Navigation
Renmin University of China(中国人民大学);Beijing Advanced Innovation Center for Future Blockchain and Privacy Computing(北京未来区块链与隐私计算先进创新中心);Horizon Robotics(地平线机器人)
ViLAM: Distilling Vision-Language Reasoning into Attention Maps for Social Robot Navigation
Dept. of Electrical and Computer Engineering, University of Maryland, College Park, MD, USA(电气与计算机工程系,马里兰大学,College Park, MD, USA);Dept. of Computer Science, University of Maryland, College Park, MD, USA(计算机科学系,马里兰大学,College Park, MD, USA);James Clark School of Engineering, University of Maryland, College Park, MD, USA(James Clark工程学院,马里兰大学,College Park, MD, USA)
$A^2$Nav: Action-Aware Zero-Shot Robot Navigation by Exploiting Vision-and-Language Ability of Foundation Models
South China University of Technology(华南理工大学);Shenzhen University(深圳大学);Pazhou Laboratory(琶洲实验室);Key Laboratory of Big Data and Intelligent Robot, Ministry of Education(教育部大数据与智能机器人重点实验室);Shenzhen Graduate School, Peking University(北京大学深圳研究生院);UMass Amherst(马萨诸塞大学阿默斯特分校);Cisco Research(思科研究院)
GuideTWSI: A Diverse Tactile Walking Surface Indicator Dataset from Synthetic and Real-World Images for Blind and Low-Vision Navigation
University of Massachusetts Amherst(马萨诸塞大学阿姆赫斯特分校);Daegu Gyeongbuk Institute of Science and Technology(大邱庆bu科学技术院);The University of Texas at Austin(德克萨斯大学奥斯汀分校);University of Maine(缅因大学)
LangMap: A Human-Verified Benchmark for Hierarchical Open-Vocabulary Goal Navigation
AIML, Adelaide University(AIML,阿德莱德大学);East China Normal University(华东师范大学);NERC-RVC, Hunan University(NERC-RVC,湖南大学);The University of Western Australia(西澳大学);Breaker Industries
Unseen from Seen: Rewriting Observation-Instruction Using Foundation Models for Augmenting Vision-Language Navigation
Shenzhen Campus of Sun Yat-sen University(中山大学深圳校区);Shanghai Jiao Tong University(上海交通大学);The University of Hong Kong(香港大学);Hunan Artificial Intelligence and Robotics Institute Company Ltd.(湖南人工智能与机器人研究院有限公司);Huawei Noah’s Ark Lab(华为诺亚实验室);Peng Cheng Laboratory(鹏城实验室)
Technical University of Munich(慕尼黑工业大学);AUDI AG(奥迪股份公司);Google(谷歌公司);Google, Zurich, Switzerland(谷歌苏黎世公司(注:严格按提取范围,机构粒度到公司级别,故保留英文全称,中文按常见译法));Department of Informatics(信息学系(注:因提取范围要求不细拆到系,此处仅作为示例说明原数据中的系名称,实际输出中应按机构粒度提取大学/公司等,故本处不单独输出该系名,仅保留大学和公司))