ETPNav: Evolving Topological Planning for Vision-Language Navigation in Continuous Environments
Center for Research on Intelligent Perception and Computing (CRIPAC), National Laboratory of Pattern Recognition (NLPR)（中国科学院大学未来技术学院、人工智能学院；模式识别国家重点实验室智能感知与计算研究中心）；School of Future Technology and School of Artificial Intelligence, University of Chinese Academy of Sciences（北京理工大学）；Beijing Institute of Technology（浙江大学）；Zhejiang University（澳大利亚国立大学）；Australian National University
Dong An, Hanqing Wang, Wenguan Wang, Zun Wang, Yan Huang, Keji He, Liang Wang
Vision-language navigation is a task that requires an agent to follow instructions to navigate in environments. It becomes increasingly crucial in the field of embodied AI, with potential applications in autonomous navigation, search and rescue, and human-robot interaction. In this paper, we propose to address a more practical yet challenging counterpart setting - vision-language navigation in continuous environments (VLN-CE). To develop a robust VLN-CE agent, we propose a new navigation framework, ETPNav, which focuses on two critical skills: 1) the capability to abstract environments and generate long-range navigation plans, and 2) the ability of obstacle-avoiding control in continuous environments. ETPNav performs online topological mapping of environments by self-organizing predicted waypoints along a traversed path, without prior environmental experience. It privileges the agent to break down the navigation procedure into high-level planning and low-level control. Concurrently, ETPNav utilizes a transformer-based cross-modal planner to generate navigation plans based on topological maps and instructions. The plan is then performed through an obstacle-avoiding controller that leverages a trial-and-error heuristic to prevent navigation from getting stuck in obstacles. Experimental results demonstrate the effectiveness of the proposed method. ETPNav yields more than 10% and 20% improvements over prior state-of-the-art on R2R-CE and RxR-CE datasets, respectively. Our code is available at https://github.com/MarSaKi/ETPNav.
NavTrust: Benchmarking Trustworthiness for Embodied Navigation
Trustworthy Autonomous Systems Laboratory at the University of California, Riverside（加州大学河滨分校可信自主系统实验室）；University of Michigan（密歇根大学）；Workday；University of Southern California（南加州大学）；Texas A&M University（德克萨斯A&M大学）；Lehigh University（莱斯大学）
The ability to perform effective planning is crucial for building an instruction-following agent. When navigating through a new environment, an agent is challenged with (1) connecting the natural language instructions with its progressively growing knowledge of the world; and (2) performing long-range planning and decision making in the form of effective exploration and error correction. Current methods are still limited on both fronts despite extensive efforts. In this paper, we introduce the Evolving Graphical Planner (EGP), a model that performs global planning for navigation based on raw s…
C-NAV: Towards Self-Evolving Continual Object Navigation in Open World
Beihang University（北京航空航天大学）；Centre for Artificial Intelligence and Robotics, HKISI-CAS（香港智能科学与工业研究院人工智能与机器人中心）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；University of Chinese Academy of Sciences（中国科学院大学）；Hangzhou International Innovation Institute, Beihang University（北京航空航天大学杭州国际创新研究院）；Beijing Jiaotong University（北京交通大学）
Faculty of Engineering, Tel-Aviv University（以色列特拉维夫大学工程学院）
Evolving Counter-Propagation Neuro-Controllers (CPNCs), rather than the traditional Feed-Forward Neuro-Controllers (FFNCs), has recently been suggested and tested using simulated robot navigation. It has been demon-strated that both convergence rate and final performance obtained by evolving CPNCs are superior to those obtained by evolving FFNCs. In this paper the maze generalization features of both types of evolved navigation controllers are examined. For this purpose the controllers are tested in an environment that drastically differs from the one used for their training. Moreover, a compa…
This paper presents the development of a Simultaneous Localization and Mapping (SLAM) based Autonomous Navigation system. The motivation for this study was to find a solution for navigating interior spaces autonomously. Interior navigation is challenging as it can be forever evolving. Solving this issue is necessary for multitude of services, like cleaning, the health industry, and in manufacturing industries. The focus of this paper is the description of the SLAM-based software architecture developed for this proposed autonomous system. A potential application of this system, oriented to a sm…
Token-Wise Latent Streaming from Slow Reasoners to Fast Planners for Dynamic Vision Language Navigation
The Hong Kong University of Science and Technology（香港科技大学）；The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；National University of Singapore（新加坡国立大学）；Zhejiang University（浙江大学）
Autonomous navigation in dynamic environment heavily depends on the environment and its topology. Prior knowledge of the environment is not usually accurate as the environment keeps evolving in time. Since robot is continuously evaluating the environment as it proceeds, deciding the optimal way to traverse the environment to get to the goal, computationally efficient yet mathematically adaptive navigation algorithms are needed. In this paper, a navigation scheme for mobile robot, capable of dealing with time variant environment is proposed. This approach consists of a global planner (A*) and l…
Data-Centric Recursive Self-Improvement for Navigation via Policy-Synchronized Language-Action Dual Bootstrapping
Shanghai AI Laboratory（上海人工智能实验室）；Shanghai Jiao Tong University（上海交通大学）；Tsinghua University（清华大学）；University of Adelaide（阿德莱德大学）；UNC Chapel Hill（北卡罗来纳大学教堂山分校）；Nanjing University（南京大学）；University of Science and Technology of China（中国科学技术大学）；Harbin Institute of Technology（哈尔滨工业大学）；Fudan University（复旦大学）
Beihang University（北京航空航天大学）；Hangzhou Innovation Institute, Beihang University（北京航空航天大学杭州创新研究院）
Vision-and-language Navigation (VLN) task requires an embodied agent to navigate to a remote location following a natural language instruction. Previous methods usually adopt a sequence model (e.g., Transformer and LSTM) as the navigator. In such a paradigm, the sequence model predicts action at each step through a maintained navigation state, which is generally represented as a one-dimensional vector. However, the crucial navigation clues (i.e., object-level environment layout) for embodied navigation task is discarded since the maintained vector is essentially unstructured. In this paper, we…
HyperDCM: Dynamic Cluster Memory Replay in Hyperbolic Space for Continual Robotic Navigation Across Scenes
Software Engineering Institute, East China Normal University（华东师范大学软件工程学院）；School of Geospatial Information, Information Engineering University（信息工程大学地理空间信息学院）；University of Shanghai for Science and Technology（上海理工大学）；Shanghai Jiao Tong University（上海交通大学）
Multi-Agent Dynamic Relational Reasoning for Social Robot Navigation
University of California, Riverside（加州大学河滨分校）；University of California, Berkeley（加州大学伯克利分校）；Korea Advanced Institute of Science and Technology（韩国科学技术院）；Stanford University（斯坦福大学）
AERR-Nav: Adaptive Exploration-Recovery-Reminiscing Strategy for Zero-Shot Object Navigation
Hong Kong Polytechnic University（香港理工大学）；Institute of automation, Chinese Academy of Sciences（中国科学院自动化研究所）；Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））
The Australian Institute for Machine Learning（澳大利亚机器学习研究所）；The University of Adelaide（阿德莱德大学）
Visual navigation is often cast as a reinforcement learning (RL) problem. Current methods typically result in a suboptimal policy that learns general obstacle avoidance and search behaviours. For example, in the target-object navigation setting, the policies learnt by traditional methods often fail to complete the task, even when the target is clearly within reach from a human perspective. In order to address this issue, we propose to learn to imagine a latent representation of the successful (sub-)goal state. To do so, we have developed a module which we call Foresight Imagination (ForeSIT). …
RAGNav: A Retrieval-Augmented Topological Reasoning Framework for Multi-Goal Visual-Language Navigation
Southwestern University of Finance（西南财经大学）；Department of Informatics,Universitat Hamburg, Hamburg, Germany（汉堡大学信息学院）；University of Electronic Science（电子科技大学）
Robot-control designers have begun to exploit the properties of the human immune system in order to produce dynamic systems that can adapt to complex, varying, real-world tasks. Jernes idiotypic-network theory has proved the most popular artificial-immune-system (AIS) method for incorporation into behaviour-based robotics, since idiotypic selection produces highly adaptive responses. However, previous efforts have mostly focused on evolving the network connections and have often worked with a single, pre-engineered set of behaviours, limiting variability. This paper describes a method for enco…
NavMorph: A Self-Evolving World Model for Vision-and-Language Navigation in Continuous Environments
State Key Laboratory of Multimodal Artificial Intelligence Systems (MAIS), Institute of Automation, Chinese Academy of Sciences (CASIA)（多模态人工智能系统国家重点实验室（MAIS）、自动化研究所、中国科学院（CASIA））；School of Artificial Intelligence, University of Chinese Academy of Sciences (UCAS)（人工智能学院、中国科学院大学（UCAS））；Peng Cheng Laboratory, ShenZhen, China（鹏城实验室、深圳中国）
Robo-Cortex: A Self-Evolving Embodied Agent via Dual-Grain Cognitive Memory and Autonomous Knowledge Induction
The Hong Kong University of Science and Technology（香港科学与技术大学）；X-Humanoid；Institute of Automation, CAS（中国科学院自动化研究所）；Beijing University of Aeronautics and Astronautics（北京航空航天大学）
AtlasVA: Self-Evolving Visual Skill Memory for Teacher-Free VLM Agents
Ant Group（蚂蚁集团）；University of Science and Technology of China（中国科学技术大学）；Westlake University（西湖大学）；University of Michigan - Ann Arbor（密歇根大学-安娜堡分校）；Sun Yat-sen University（中山大学）
