Object-Goal Navigation (ObjectNav) requires an agent to find and navigate to a target object category in unknown environments. While recent Large Language Model (LLM)-based agents exhibit zero-shot reasoning, they often rely on a "reactive" paradigm that lacks explicit spatial memory, leading to redundant exploration and myopic behaviors. To address these limitations, we propose a transition from reactive AI to "Map-Based AI" by integrating LLM-based semantic inference with a hybrid topological-grid mapping system. Our framework employs a fine-tuned Llama-2 model via Low-Rank Adaptation (LoRA) to infer semantic zone categories and target existence probabilities from verbalized object observations. In this study, a "zone" is defined as a functional area described by the set of observed objects, providing crucial semantic co-occurrence cues for finding the target. This semantic information is integrated into a topological graph, enabling the agent to prioritize high-probability areas and perform systematic exploration via Traveling Salesman Problem (TSP) optimization. Evaluations in the AI2-THOR simulator demonstrate that our approach significantly outperforms traditional frontier exploration and reactive LLM baselines, achieving a superior Success Rate (SR) and Success weighted by Path Length (SPL).
Swiss Federal School of Technology in Lausanne - EPFL（洛桑联邦理工学院）；University of Tsukuba（筑波大学）；Swiss Federal School of Technology in Zurich - ETH Zurich（苏黎世联邦理工学院）
Autonomous navigation in highly populated areas remains a challenging task for robots because of the difficulty in guaranteeing safe interactions with pedestrians in unstructured situations. In this work, we present a crowd navigation control framework that delivers continuous obstacle avoidance and post-contact control evaluated on an autonomous personal mobility vehicle. We propose evaluation metrics for accounting efficiency, controller response and crowd interactions in natural crowds. We report the results of over 110 trials in different crowd types: sparse, flows, and mixed traffic, with…
Key Lab of Intelligent Information Processing Laboratory of the Chinese Academy of Sciences (CAS)（中国科学院智能信息处理重点实验室）；Huawei Application Innovate Laboratory, Beijing（华为应用创新实验室（北京））；Institute of Intelligent Computing Technology, Suzhou, CAS（中国科学院苏州智能计算技术研究所）；Institute of Computing Technology（中国科学院计算技术研究所）；Beijing University of Chinese Academy of Sciences（中国科学院大学）
The goal of object navigation is to reach the expected objects according to visual information in the unseen environments. Previous works usually implement deep models to train an agent to predict actions in real-time. However, in the unseen environment, when the target object is not in egocentric view, the agent may not be able to make wise decisions due to the lack of guidance. In this paper, we propose a hierarchical object-to-zone (HOZ) graph to guide the agent in a coarse-to-fine manner, and an online-learning mechanism is also proposed to update HOZ according to the real-time observation…
Bristol Robotics Laboratory, The University of Bristol（布里斯托尔机器人实验室，布里斯托尔大学）；Visual Information Laboratory, The University of Bristol（视觉信息实验室，布里斯托尔大学）；Department of Electrical and Electronic Engineering, The University of Manchester（曼彻斯特大学电气与电子工程系）；The Department of Computer Science and Engineering & The Department of Ophthalmology, The University of California, San Diego（加利福尼亚大学圣地亚哥分校计算机科学与工程系及眼科系）；Hangzhou ATG Intelligent Technology Co. Ltd.（杭州ATG智能科技有限公司）；Amazon（亚马逊公司）；The University of California, San Diego（加利福尼亚大学圣地亚哥分校）；The University of Manchester（曼彻斯特大学）；The University of Bristol（布里斯托尔大学）；Amazon, Seattle, USA（亚马逊公司（西雅图））；The Department of Ophthalmology, The University of California, San Diego（加利福尼亚大学圣地亚哥分校眼科系）
This paper presents an agile reactive navigation strategy for driving a non-holonomic ground vehicle around a preset course of gates in a cluttered environment using a low-cost processor array sensor. This enables machine vision tasks to be performed directly upon the sensor's image plane, rather than using a separate general-purpose computer. We demonstrate a small ground vehicle running through or avoiding multiple gates at high speed using minimal computational resources. To achieve this, target tracking algorithms are developed for the Pixel Processing Array and captured images are then pr…
Instituto de Investigación en Ingeniería de Aragón, (I3A). Universidad de Zaragoza（阿拉贡工程研究所（I3A），萨拉戈萨大学）
The visual functions of visual prostheses such as field of view, resolution and dynamic range, seriously restrict the person's ability to navigate in unknown environments. Implanted patients still require constant assistance for navigating from one location to another. Hence, there is a need for a system that is able to assist them safely during their journey. In this work, we propose an augmented reality navigation system for visual prosthesis that incorporates a software of reactive navigation and path planning which guides the subject through convenient, obstacle-free route. It consists on …
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；University of Chinese Academy of Sciences（中国科学院大学）；Huawei Noah’s Ark Lab（华为诺亚方舟实验室）
Visual navigation in complex environments is inefficient with traditional reactive policy or general-purposed recurrent policy. To address the long-term memory issue, this paper proposes a graph attention memory (GAM) architecture consisting of memory construction module, graph attention module and control module. The memory construction module builds the topological graph based on supervised learning by taking the exploration prior. Then, guided attention features are extracted with the graph attention module. Finally, the deep reinforcement learning based control module makes decisions based…
ForeFly: A Dual-Horizon World Action Model for Aerial Vision-Language Navigation
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；University of Chinese Academy of Sciences（中国科学院大学）；Peng Cheng Laboratory（鹏城实验室）；Duke Kunshan University（昆山杜克大学）
Multi-Platform Teach-and-Repeat Navigation by Visual Place Recognition Based on Deep-Learned Local Features
Czech Institute of Informatics, Robotics and Cybernetics, Czech Technical University in Prague（捷克布拉格技术大学信息学、机器人学与控制论研究所）；Faculty of Electrical Engineering, Czech Technical University in Prague（捷克布拉格技术大学电气工程学院）
Dorabot Inc.（Dorabot公司）；City University of Hong Kong（香港城市大学）；Fuzhou University（福州大学）
High-speed, low-latency obstacle avoidance that is insensitive to sensor noise is essential for enabling multiple decentralized robots to function reliably in cluttered and dynamic environments. While other distributed multi-agent collision avoidance systems exist, these systems require online geometric optimization where tedious parameter tuning and perfect sensing are necessary. We present a novel end-to-end framework to generate reactive collision avoidance policy for efficient distributed multi-agent navigation. Our method formulates an agent's navigation strategy as a deep neural network …
Rectilinear forms of snake-like robotic locomotion are anticipated to be an advantage in obstacle-strewn scenarios characterizing urban disaster zones, subterranean collapses, and other natural environments. The elongated, laterally-narrow footprint associated with these motion strategies is well-suited to traversal of confined spaces and narrow pathways. Navigation and path planning in the absence of global sensing, however, remains a pivotal challenge to be addressed prior to practical deployment of these robotic mechanisms. Several challenges related to visual processing and localization ne…
The role of unmanned vehicles for searching and localizing the victims in disaster impacted areas such as earthquake-struck zones is getting more important. Self-navigation on an earthquake zone has a unique challenge of detecting irregularly shaped obstacles such as road cracks, debris on the streets, and water puddles. In this paper, we characterize a number of state-of-the-art FCN models on mobile embedded platforms for self-navigation at these sites containing extremely irregular obstacles. We evaluate the models in terms of accuracy, performance, and energy efficiency. We present a few op…
RVN-Bench: A Benchmark for Reactive Visual Navigation
Department of Electrical and Computer Engineering, Seoul National University (SNU) and Automation and Systems Research Institute (ASRI) and Sequor Robotics Inc.（电气与计算机工程系，首尔国立大学（SNU）和自动化与系统研究所（ASRI）以及Sequor机器人公司）；Department of Electrical and Computer Engineering, Seoul National University (SNU) and Automation and Systems Research Institute (ASRI)（电气与计算机工程系，首尔国立大学（SNU）和自动化与系统研究所（ASRI））；Interdisciplinary Program in Artificial Intelligence, Seoul National University (SNU) and Automation and Systems Research Institute (ASRI) and Sequor Robotics Inc.（人工智能跨学科项目，首尔国立大学（SNU）和自动化与系统研究所（ASRI）以及Sequor机器人公司）；Sequor Robotics Inc.（Sequor机器人公司）
We focus on the utilisation of reactive trajectory imitation controllers for goal-directed mobile robot navigation. We propose a topological navigation graph (TNG) - an imitation-learning-based framework for navigating through environments with intersecting trajectories. The TNG framework represents the environment as a directed graph composed of deep neural networks. Each vertex of the graph corresponds to a trajectory and is represented by a trajectory identification classifier and a trajectory imitation controller. For trajectory following, we propose the novel use of neural object detectio…
