Multi-robot navigation is a challenging task in which multiple robots must be coordinated simultaneously within dynamic environments. We apply deep reinforcement learning (DRL) to learn a decentralized end-to-end policy which maps raw sensor data to the command velocities of the agent. In order to enable the policy to generalize, the training is performed in different environments and scenarios. The learned policy is tested and evaluated in common multi-robot scenarios like switching a place, an intersection and a bottleneck situation. This policy allows the agent to recover from dead ends and to navigate through complex environments.
EnQuery: Ensemble Policies for Diverse Query-Generation in Preference Alignment of Robot Navigation
University of Bonn（波恩大学）；Center for Robotics at the University of Bonn（波恩大学机器人中心）；Lamarr Institute for Machine Learning and Artificial Intelligence（拉马尔机器学习与人工智能研究所）
FlashNav: Training Deployable Robot Navigation Policies in Seconds
Eastern Institute of Technology, Ningbo（宁波东方理工大学）；The Hong Kong Polytechnic University（香港理工大学）；National University of Singapore（新加坡国立大学）；University of Science and Technology of China（中国科学技术大学）；Shanghai Jiao Tong University（上海交通大学）
Data Scaling for Navigation in Unknown Environments
Signal Processing Research Centre, Faculty of Information Technology and Communication Sciences, Tampere University（信号处理研究中心，信息科技与通信科学学院，塔尔皮奥大学）；Graduate School of Information Sciences, Tohoku University（信息科学研究生院，东北大学）；Faculty of Engineering, Turku University of Applied Sciences（工程学院，图尔库应用科学大学）
Colorado School of Mines（科罗拉多矿业学院）；DEVCOM Army Research Laboratory（DEVCOM陆军研究实验室）；University of Massachusetts Amherst（马萨诸塞大学阿默斯特分校）
When robots operate in real-world off-road environments with unstructured terrains, the ability to adapt their navigational policy is critical for effective and safe navigation. However, off-road terrains introduce several challenges to robot navigation, including dynamic obstacles and terrain uncertainty, leading to inefficient traversal or navigation failures. To address these challenges, we introduce a novel approach for adaptation by negotiation that enables a ground robot to adjust its navigational behaviors through a negotiation process. Our approach first learns prediction models for va…
Queensland University of Technology (QUT)（昆士兰科技大学（QUT））
Emerging object-based SLAM algorithms can build a graph representation of an environment comprising nodes for robot poses and object landmarks. However, while this map will contain static objects such as furniture or appliances, many moveable objects (e.g. the car keys, the glasses, or a magazine), are not suitable as landmarks and will not be part of the map due to their non-static nature. We show that Graph Convolutional Networks can learn navigation policies to find such unmapped objects by learning to exploit the hidden probabilistic model that governs where these objects appear in the env…
Learned Neural Network based policies have shown promising results for robot navigation. However, most of these approaches fall short of being used on a real robot due to the extensive simulated training they require. These simulations lack the visuals and dynamics of the real world, which makes it infeasible to deploy on a real robot. We present a novel Neural Net based policy, NavNet, which allows for easy deployment on a real robot. It consists of two sub policies -- a high level policy which can understand real images and perform long range planning expressed in high level commands; a low …
OrangeDev s.r.l., Firenze, Italy（OrangeDev s.r.l.，意大利佛罗伦萨）；Cognitive Robotics and Social Sensing LaboratoryICAR-CNR, Palermo, Italy（意大利帕勒莫ICAR-CNR认知机器人与社会感知实验室）
Visual navigation models based on deep learning can learn effective policies when trained on large amounts of visual observations through reinforcement learning. Unfortunately, collecting the required experience in the real world requires the deployment of a robotic platform, which is expensive and time-consuming. To deal with this limitation, several simulation platforms have been proposed in order to train visual navigation policies on virtual environments efficiently. Despite the advantages they offer, simulators present a limited realism in terms of appearance and physical dynamics, leadin…
In this paper, a novel deep reinforcement learning (DRL)-based method is proposed to navigate the robot team through unknown complex environments, where the geometric centroid of the robot team aims to reach the goal position while avoiding collisions and maintaining connectivity. Decentralized robot-level policies are derived using a mechanism of centralized learning and decentralized executing. The proposed method can derive end-to-end policies, which map raw lidar measurements into velocity control commands of robots without the necessity of constructing obstacle maps. Simulation and indoor…
Advances in visual navigation methods have led to intelligent embodied navigation agents capable of learning meaningful representations from raw RGB images and perform a wide variety of tasks involving structural and semantic reasoning. However, most learning-based navigation policies are trained and tested in simulation environments. In order for these policies to be practically useful, they need to be transferred to the real-world. In this paper, we propose an unsupervised domain adaptation method for visual navigation. Our method translates the images in the target domain to the source doma…
Carnegie Mellon University（卡内基梅隆大学）；Chinese Academy of Sciences（中国科学院）；Lebanese University（黎巴嫩大学）
Recent literature in the robotics community has focused on learning robot behaviors that abstract out lower-level details of robot control. To fully leverage the efficacy of such behaviors, it is necessary to select and sequence them to achieve a given task. In this paper, we present an approach to both learn and sequence robot behaviors, applied to the problem of visual navigation of mobile robots. We construct a layered representation of control policies composed of low- level behaviors and a meta-level policy. The low-level behaviors enable the robot to locomote in a particular environment …
Johns Hopkins University（约翰斯·霍普金斯大学）；Johns Hopkins University Applied Physics Lab（约翰斯·霍普金斯大学应用物理实验室）；UC Berkeley（加州大学伯克利分校）
Human-aware robot navigation promises a range of applications in which mobile robots bring versatile assistance to people in common human environments. While prior research has mostly focused on modeling pedestrians as independent, intentional individuals, people move in groups; consequently, it is imperative for mobile robots to respect human groups when navigating around people. This paper explores learning group-aware navigation policies based on dynamic group formation using deep reinforcement learning. Through simulation experiments, we show that group-aware policies, compared to baseline…
Robotics Institute, Carnegie Mellon University（卡内基梅隆大学机器人研究所）
Multisensory polices are known to enhance both state estimation and target tracking. However, in the space of end-to-end sensorimotor control, this multi-sensor outlook has received limited attention. Moreover, systematic ways to make policies robust to partial sensor failure are not well explored. In this work, we propose a specific customization of Dropout, called \textit{Sensor Dropout}, to improve multisensory policy robustness and handle partial failure in the sensor-set. We also introduce an additional auxiliary loss on the policy network in order to reduce variance in the band of potent…
FlowNav: Combining Flow Matching and Depth Priors for Efficient Navigation
Artificial Intelligence and Robotics Lab, Department of Computer Science and Artificial Intelligence, University of Technology Nuremberg（人工智能与机器人实验室，计算机科学与人工智能系，图腾大学）；Inria, Ecole Normale Supérieure, CNRS, PSL Research University（法国国家信息与自动化研究所，巴黎高等师范学校，国家科学研究中心，巴黎综合理工研究学院）
We present a novel sensor-based learning navigation algorithm to compute a collision-free trajectory for a robot in dense and dynamic environments with moving obstacles or targets. Our approach uses deep reinforcement learning-based expert policy that is trained using a sim2real paradigm. In order to increase the reliability and handle the failure cases of the expert policy, we combine with a policy extraction technique to transform the resulting policy into a decision tree format. The resulting decision tree has properties which we use to analyze and modify the policy and improve performance …
University of Toronto Robotics Institute（多伦多大学机器人研究所）
The robustness of visual navigation policies trained through imitation often hinges on the augmentation of the training image-action pairs. Traditionally, this has been done by collecting data from multiple cameras, by using standard data augmentations from computer vision, such as adding random noise to each image, or by synthesizing training images. In this paper we show that there is another practical alternative for data augmentation for visual navigation based on extrapolating viewpoint embeddings and actions nearby the ones observed in the training data. Our method makes use of the geome…
Learning Social Navigation from Positive and Negative Demonstrations and Rule-Based Specifications
Department of Artificial Intelligence, Korea University（人工智能系，韩国大学）；Department of Computer Science and Engineering（计算机科学与工程系）；Department of Artificial Intelligence, Yonsei University（人工智能系，yonsei大学）；Robotics Institute, School of Computer Science at Carnegie Mellon University（机器人研究所，卡内基梅隆大学计算机科学学院）；Mobinn；Department of Electrical and Computer Engineering（电气与计算机工程系）；Department of Statistics（统计学系）
Seeing through Uncertainty: Robust Task-Oriented Optimization in Visual Navigation
Shanghai Jiao Tong University（上海交通大学）；School of Automation and Intelligent Sensing（自动化与智能感知学院）；Key Laboratory of System Control and Information Processing（系统控制与信息处理重点实验室）；Ministry of Education of China（中华人民共和国教育部）；National Key Laboratory of Human-Machine Hybrid Augmented Intelligence（人机混合增强智能国家级重点实验室）；Institute of Artificial Intelligence and Robotics（人工智能与机器人研究院）
The purpose of navigation is to determine the position, velocity, and orientation of manned and autonomous platforms, humans, and animals. Obtaining accurate navigation commonly requires fusion between several sensors, such as inertial sensors and global navigation satellite systems, in a model-based, nonlinear estimation framework. Recently, data-driven approaches applied in various fields show state-of-the-art performance, compared to model-based methods. In this paper we review multidisciplinary, data-driven based navigation algorithms developed and experimentally proven at the Autonomous N…
Toshiba Europe Limited（东芝欧洲有限公司）；University of Cambridge（剑桥大学）
We present NavACL, a method of automatic curriculum learning tailored to the navigation task. NavACL is simple to train and efficiently selects relevant tasks using geometric features. In our experiments, deep reinforcement learning agents trained using NavACL significantly outperform state-of-the-art agents trained with uniform sampling -- the current standard. Furthermore, our agents can navigate through unknown cluttered indoor environments to semantically-specified targets using only RGB images. Obstacle-avoiding policies and frozen feature networks support transfer to unseen real-world en…
We present a novel approach to improve the performance of deep reinforcement learning (DRL) based outdoor robot navigation systems. Most, existing DRL methods are based on carefully designed dense reward functions that learn the efficient behavior in an environment. We circumvent this issue by working only with sparse rewards (which are easy to design), and propose a novel adaptive Heavy-Tailed Reinforce algorithm for Outdoor Navigation called HTRON. Our main idea is to utilize heavy-tailed policy parametrizations which implicitly induce exploration in sparse reward settings. We evaluate the p…
Robots moving safely and in a socially compliant manner in dynamic human environments is an essential benchmark for long-term robot autonomy. However, it is not feasible to learn and benchmark social navigation behaviors entirely in the real world, as learning is data-intensive, and it is challenging to make safety guarantees during training. Therefore, simulation-based benchmarks that provide abstractions for social navigation are required. A framework for these benchmarks would need to support a wide variety of learning approaches, be extensible to the broad range of social navigation scenar…
NavWM: A Unified Navigation World Model for Foresight-Driven Planning
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；Beihang University（北京航空航天大学）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）
Meta AI Research（Meta人工智能研究院）；Carnegie Mellon University（卡内基梅隆大学）；University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
Realistic long-horizon tasks like image-goal navigation involve exploratory and exploitative phases. Assigned with an image of the goal, an embodied agent must explore to discover the goal, i.e., search efficiently using learned priors. Once the goal is discovered, the agent must accurately calibrate the last-mile of navigation to the goal. As with any robust system, switches between exploratory goal discovery and exploitative last-mile navigation enable better recovery from errors. Following these intuitive guide rails, we propose SLING to improve the performance of existing image-goal naviga…
Navigating fluently around pedestrians is a necessary capability for mobile robots deployed in human environments, such as buildings and homes. While research on social navigation has focused mainly on the scalability with the number of pedestrians in open spaces, typical indoor environments present the additional challenge of constrained spaces such as corridors and doorways that limit maneuverability and influence patterns of pedestrian interaction. We present an approach based on reinforcement learning (RL) to learn policies capable of dynamic adaptation to the presence of moving pedestrian…
RoboRAN: A Unified Robotics Framework for Reinforcement Learning-Based Autonomous Navigation
Space Robotics Research Group, SnT University of Luxembourg（锡尔特大学卢森堡大学空间机器人研究组）；Georgia Tech-Europe IRL2958（佐治亚理工学院欧洲分校IRL2958）；GT-CNRS, Metz, France（GT-CNRS，法国梅茨）；Earth Species Project（地球物种项目）
ReaDy-Go: Real-to-Sim Dynamic 3D Gaussian Splatting Simulation for Environment-Specific Visual Navigation with Moving Obstacles
Department of Aerospace Engineering, Seoul National University（首尔国立大学航空航天工程系）；InnoCORE AI-Transformed Aerospace Research Center, KAIST（韩国科学技术院（KAIST）人工智能赋能航空航天研究中心）
Learning agents can optimize standard autonomous navigation improving flexibility, efficiency, and computational cost of the system by adopting a wide variety of approaches. This work introduces the \textit{PIC4rl-gym}, a fundamental modular framework to enhance navigation and learning research by mixing ROS2 and Gazebo, the standard tools of the robotics community, with Deep Reinforcement Learning (DRL). The paper describes the whole structure of the PIC4rl-gym, which fully integrates DRL agent's training and testing in several indoor and outdoor navigation scenarios and tasks. A modular appr…
Prior Does Matter: Visual Navigation via Denoising Diffusion Bridge Models
School of Computer Science and Engineering, Sun Yat-sen University（中山大学计算机科学与工程学院）；School of Intelligent Systems Engineering, Sun Yat-sen University（中山大学智能系统工程学院）
Queensland University of Technology（昆士兰科技大学）；Universidad Nacional de Ingeniería（国立工程大学）
Visual navigation tasks in real-world environments often require both self-motion and place recognition feedback. While deep reinforcement learning has shown success in solving these perception and decision-making problems in an end-to-end manner, these algorithms require large amounts of experience to learn navigation policies from high-dimensional data, which is generally impractical for real robots due to sample complexity. In this paper, we address these problems with two main contributions. We first leverage place recognition and deep learning techniques combined with goal destination fee…
Understanding and following directions provided by humans can enable robots to navigate effectively in unknown situations. We present FollowNet, an end-to-end differentiable neural architecture for learning multi-modal navigation policies. FollowNet maps natural language instructions as well as visual and depth inputs to locomotion primitives. FollowNet processes instructions using an attention mechanism conditioned on its visual and depth input to focus on the relevant parts of the command while performing the navigation task. Deep reinforcement learning (RL) a sparse reward learns simultaneo…
GeoScaffold: Learning Compact Geometric Latents via Reconstruction for Efficient Vision-Language Navigation
Nanjing University（南京大学）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；Nanjing University of Aeronautics and Astronautics（南京航空航天大学）；MAICRO（迈克罗）
Johns Hopkins University Applied Physics Laboratory（约翰斯·霍普金斯大学应用物理实验室）
Robot navigation traditionally relies on building an explicit map that is used to plan collision-free trajectories to a desired target. In deformable, complex terrain, using geometric-based approaches can fail to find a path due to mischaracterizing deformable objects as rigid and impassable. Instead, we learn to predict an estimate of traversability of terrain regions and to prefer regions that are easier to navigate (e.g., short grass over small shrubs). Rather than predicting collisions, we instead regress on realized error compared to a canonical dynamics model. We train with an on-policy …
LDP: A Local Diffusion Planner for Efficient Robot Navigation and Collision Avoidance
Institute of Advanced Technology, University of Science and Technology of China（中国科学技术大学先进技术研究院）；School of Data Science, USTC（中国科学技术大学数据科学学院）；School of Computer Science and Technology, USTC（中国科学技术大学计算机科学与技术学院）
