Object-goal navigation in open-vocabulary settings requires agents to locate novel objects in unseen environments, yet existing approaches suffer from opaque decision-making processes and low success rate on locating unseen objects. To address these challenges, we propose Nav-$R^2$, a framework that explicitly models two critical types of relationships, target-environment modeling and environment-action planning, through structured Chain-of-Thought (CoT) reasoning coupled with a Similarity-Aware Memory. We construct a Nav$R^2$-CoT dataset that teaches the model to perceive the environment, focus on target-related objects in the surrounding context and finally make future action plans. Our SA-Mem preserves the most target-relevant and current observation-relevant features from both temporal and semantic perspectives by compressing video frames and fusing historical observations, while introducing no additional parameters. Compared to previous methods, Nav-R^2 achieves state-of-the-art performance in localizing unseen objects through a streamlined and efficient pipeline, avoiding overfitting to seen object categories while maintaining real-time inference at 2Hz. Resources will be made publicly available at \href{https://github.com/AMAP-EAI/Nav-R2}{github link}.
Metis: A Generalizable and Efficient World-Action Model for Autonomous Driving and Urban Navigation
Fudan University（复旦大学）；Shanghai Innovation Institute（上海创新研究院）；The University of Hong Kong（香港大学）；Tongji University（同济大学）；Li Auto Inc.（理想汽车）；Huazhong University of Science and Technology（华中科技大学）；Imperial College London（伦敦帝国理工学院）；University of Surrey（萨里大学）
DOZE: A Dataset for Open-Vocabulary Zero-Shot Object Navigation in Dynamic Environments
College of Engineering, Peking University（北京大学工学院）；School of Computer Science, Peking University（北京大学计算机学院）；The University of Hong Kong（香港大学）；The Hong Kong University of Science and Technology（香港科技大学）
Distilling LLM Prior to Flow Model for Generalizable Agent's Imagination in Object Goal Navigation
Sun Yat-sen University（中山大学）；The University of Hong Kong（香港大学）；Key Laboratory of Machine Intelligence and Advanced Computing, Ministry of Education（教育部机器智能与高级计算重点实验室）
MSGNav: Unleashing the Power of Multi-modal 3D Scene Graph for Zero-Shot Embodied Navigation
Fujian Key Laboratory of Urban Intelligent Sensing and Computing（福建智能感知与计算重点实验室）；Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China（多媒体可信感知与高效计算重点实验室）；Beihang University（北航）；Nanyang Technological University（南洋理工大学）；University of Chinese Academy of Sciences（中国科学院大学）；Zhongguancun Academy（中关村学院）
AetherVision-Bench: An Open-Vocabulary RGB-Infrared Benchmark for Multi-Angle Segmentation across Aerial and Ground Perspectives
Robert Bosch Centre for Cyber Physical Systems, Indian Institute of Science, Bengaluru, India（罗伯特·博世协同物理系统中心，印度科学研究院，班加罗尔）；Kotak IISc AI-ML Centre, Indian Institute of Science, Bengaluru, India（Kotak IISc AI-ML 中心，印度科学研究院，班加罗尔）；Department of Aerospace Engineering, Indian Institute of Science, Bengaluru, India（航空航天工程系，印度科学研究院，班加罗尔）
XOV-Action: Towards Generalizable Open-Vocabulary Action Recognition
School of Computer Science and Engineering, Sun Yat-sen University（中山大学计算机科学与工程学院）；Institute of Big Data, Fudan University（复旦大学大数据研究院）；AI Thrust, Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州）人工智能方向）；College of Computing and Data Science, Nanyang Technological University（南洋理工大学计算机与数据科学学院）
Columbia University（哥伦比亚大学）；University of Washington（华盛顿大学）
For robots to be generally useful, they must be able to find arbitrary objects described by people (i.e., be language-driven) even without expensive navigation training on in-domain data (i.e., perform zero-shot inference). We explore these capabilities in a unified setting: language-driven zero-shot object navigation (L-ZSON). Inspired by the recent success of open-vocabulary models for image classification, we investigate a straightforward framework, CLIP on Wheels (CoW), to adapt open-vocabulary models to this task without fine-tuning. To better evaluate L-ZSON, we introduce the Pasture ben…
OVER-NAV: Elevating Iterative Vision-and-Language Navigation with Open-Vocabulary Detection and StructurEd Representation
The University of Hong Kong（香港大学）；Sun Yat-sen University（中山大学）；Digital Content Technology Center, Tencent Games（腾讯游戏数字内容技术中心）；GuangDong Province Key Laboratory of Information Security Technology（广东省信息安全重点实验室）
In Vision-and-Language Navigation (VLN), an agent needs to navigate through the environment based on natural language instructions. Due to limited available data for agent training and finite diversity in navigation environments, it is challenging for the agent to generalize to new, unseen environments. To address this problem, we propose EnvEdit, a data augmentation method that creates new environments by editing existing environments, which are used to train a more generalizable agent. Our augmented environments can differ from the seen environments in three diverse aspects: style, object ap…
University of California, Santa Barbara（加州大学圣巴巴拉分校）
Existing research studies on vision and language grounding for robot navigation focus on improving model-free deep reinforcement learning (DRL) models in synthetic environments. However, model-free DRL models do not consider the dynamics in the real-world environments, and they often fail to generalize to new scenes. In this paper, we take a radical approach to bridge the gap between synthetic studies and real-world practices---We propose a novel, planned-ahead hybrid reinforcement learning model that combines model-free and model-based reinforcement learning to solve a real-world vision-langu…
University of Freiburg（弗赖堡大学）；Google Research（谷歌研究院）；University of Technology Nuremberg（纽伦堡工业大学）
While interacting in the world is a multi-sensory experience, many robots continue to predominantly rely on visual perception to map and navigate in their environments. In this work, we propose Audio-Visual-Language Maps (AVLMaps), a unified 3D spatial map representation for storing cross-modal information from audio, visual, and language cues. AVLMaps integrate the open-vocabulary capabilities of multimodal foundation models pre-trained on Internet-scale data by fusing their features into a centralized 3D voxel grid. In the context of navigation, we show that AVLMaps enable robot systems to i…
A Hierarchical Semantic Distillation Framework for Open-Vocabulary Object Detection
School of Computer Science and Engineering, Sun Yat-sen University（中山大学计算机科学与工程学院）；Peng Cheng Laboratory（鹏城实验室）；Tongyi Lab, Alibaba Group（阿里巴巴集团通义实验室）；Guangdong Key Laboratory of Information Security Technology, Sun Yat-sen University（中山大学广东省信息安全技术重点实验室）；Key Laboratory of Machine Intelligence and Advanced Computing, Sun Yat-sen University, Ministry of Education（中山大学机器智能与先进计算教育部重点实验室）
OVGaussian: Generalizable 3D Gaussian Segmentation with Open Vocabularies
The University of Sydney（悉尼大学）；Fudan University（复旦大学）；Shanghai AI Laboratory（上海人工智能实验室）；The University of Hong Kong（香港大学）；National University of Singapore（新加坡国立大学）；Imperial College London（伦敦帝国学院）；The University of Melbourne（墨尔本大学）；Texas A&M University（德克萨斯农工大学）
UnitedVLN: Generalizable Gaussian Splatting for Continuous Vision-Language Navigation
Nanjing University of Science and Technology（南京理工大学）；Northwest Polytechnical University（西北工业大学）；The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））；Tsinghua University（清华大学）
Towards Domain-Generalized Open-Vocabulary Object Detection: A Progressive Domain-invariant Cross-modal Alignment Method
MAIS, Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所多模态人工智能系统实验室）；Institute of Microelectronics, University of Chinese Academy of Sciences（中国科学院大学微电子学院）；School of Advanced Interdisciplinary Sciences, University of Chinese Academy of Sciences（中国科学院大学前沿交叉科学学院）
Jet Propulsion Laboratory（喷气推进实验室）；California Institute of Technology（加州理工学院）；Swiss Federal Institute of Technology（瑞士联邦理工学院）；ETH Zürich（苏黎世联邦理工学院）；FieldAI Inc.（FieldAI公司）；Stanford University（斯坦福大学）；University of California Berkeley（加州大学伯克利分校）
FlightGPT: Towards Generalizable and Interpretable UAV Vision-and-Language Navigation with Vision-Language Models
School of Intelligent Systems Engineering, Sun Yat-Sen University（中山大学智能系统工程学院）；DP Technology（DP技术公司）；Beijing University Of Posts and Telecommunications（北京邮电大学）；Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）；Tongji University（同济大学）；School of Integrated Innovation, Chulalongkorn University（朱拉隆功大学创新学院）
OmniObject3D: Large-Vocabulary 3D Object Dataset for Realistic Perception, Reconstruction and Generation
The Chinese University of Hong Kong（香港中文大学）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；SenseTime Research（商汤科技研究院）；Hong Kong University of Science and Technology（香港科技大学）；Nanyang Technological University（南洋理工大学）
LangMap: A Human-Verified Benchmark for Hierarchical Open-Vocabulary Goal Navigation
AIML, Adelaide University（AIML，阿德莱德大学）；East China Normal University（华东师范大学）；NERC-RVC, Hunan University（NERC-RVC，湖南大学）；The University of Western Australia（西澳大学）；Breaker Industries
Arizona State University（亚利桑那州立大学）；Adobe Systems, Inc.（Adobe系统公司）
We study the problem of learning a generalizable action policy for an intelligent agent to actively approach an object of interest in an indoor environment solely from its visual inputs. While scene-driven or recognition-driven visual navigation has been widely studied, prior efforts suffer severely from the limited generalization capability. In this paper, we first argue the object searching task is environment dependent while the approaching ability is general. To learn a generalizable approaching policy, we present a novel solution dubbed as GAPLE which adopts two channels of visual feature…
