Autonomous robotic navigation in real-world environments requires exploration to acquire environmental information as well as goal-directed navigation in order to reach specified targets. Active inference (AIF) based on the free-energy principle provides a unified framework for these behaviors by minimizing the expected free energy (EFE), thereby combining epistemic and extrinsic values. To realize this practically, we propose a deep AIF framework that integrates a diffusion policy as the policy model and a multiple timescale recurrent state-space model (MTRSSM) as the world model. The diffusion policy generates diverse candidate actions while the MTRSSM predicts their long-horizon consequences through latent imagination, enabling action selection that minimizes EFE. Real-world navigation experiments, including baseline comparisons, component ablations, and robustness evaluations, demonstrated that our framework achieved higher success rates and fewer collisions, particularly in exploration-demanding scenarios. These results highlight how AIF based on EFE minimization can unify exploration and goal-directed navigation in real-world robotic settings.
Albert Ludwig University of Freiburg（阿尔伯特-路德维希-弗莱堡大学）；The Hong Kong University of Science and Technology（香港科技大学）
This paper investigates exploration strategies of Deep Reinforcement Learning (DRL) methods to learn navigation policies for mobile robots. In particular, we augment the normal external reward for training DRL algorithms with intrinsic reward signals measured by curiosity. We test our approach in a mapless navigation setting, where the autonomous agent is required to navigate without the occupancy map of the environment, to targets whose relative locations can be easily acquired through low-cost solutions (e.g., visible light localization, Wi-Fi signal localization). We validate that the intri…
Carnegie Mellon University（卡内基梅隆大学）；Facebook AI Research（Facebook人工智能研究）
Numerous past works have tackled the problem of task-driven navigation. But, how to effectively explore a new environment to enable a variety of down-stream tasks has received much less attention. In this work, we study how agents can autonomously explore realistic and complex 3D environments without the context of task-rewards. We propose a learning-based approach and investigate different policy architectures, reward functions, and training paradigms. We find that the use of policies with spatial memory that are bootstrapped with imitation learning and finally finetuned with coverage rewards…
TopoNav: Topological Navigation for Efficient Exploration in Sparse Reward Environments
University of Maryland, Baltimore County（马里兰大学巴尔的摩县分校）；Amazon Inc.（亚马逊公司）；DEVCOM Army Research Lab（DEVCOM陆军研究实验室）；Stormfish Scientific Corporation（斯托姆菲什科学公司）
Confidence-Controlled Exploration: Efficient Sparse-Reward Policy Learning for Robot Navigation
U.S. Army Research Laboratory（美国陆军研究实验室）；JP Morgan AI Research（摩根大通人工智能研究院）；University of Texas at Austin（德克萨斯大学奥斯汀分校）；University of Central Florida（中佛罗里达大学）
Intrinsic-Motivation Multi-Robot Social Formation Navigation with Coordinated Exploration
School of Computer Science and Technology（计算机科学与技术学院）；Wuhan University of Science and Technology（武汉科技大学）；Hubei Province Key Laboratory of Intelligent Information Processing（湖北省智能信息处理重点实验室）
Artificial Intelligence Research Institute, Federal Research Center for Computer Science and Control of Russian Academy of Sciences（俄罗斯科学院计算机科学与控制联邦研究中心人工智能研究所）；Moscow Institute of Physics and Technology（莫斯科物理技术学院）
Exploration is an important step in autonomous navigation of robotic systems. In this paper we introduce a series of enhancements for exploration algorithms in order to use them with vision-based simultaneous localization and mapping (vSLAM) methods. We evaluate developed approaches in photo-realistic simulator in two modes: with ground-truth depths and neural network reconstructed depth maps as vSLAM input. We evaluate standard metrics in order to estimate exploration coverage.
The City University of New York（纽约市立大学）；Hunter College（亨特学院）；The Graduate Center of The City University of New York（纽约市立大学研究生中心）
To perform tasks well in a new domain, one must first know something about it. This paper reports on a robot controller for navigation through unfamiliar indoor worlds. Based on spatial affordances, it integrates planning with reactive heuristics. Before it addresses specific targets, however, the system deliberately explores for high-level connectivity and captures that data in a cognitive spatial model. Despite limited exploration time, planning in the resultant model is faster and better supports successful travel in a challenging, realistic space.
This work studies the problem of object goal navigation which involves navigating to an instance of the given object category in unseen environments. End-to-end learning-based navigation methods struggle at this task as they are ineffective at exploration and long-term planning. We propose a modular system called, `Goal-Oriented Semantic Exploration' which builds an episodic semantic map and uses it to explore the environment efficiently based on the goal object category. Empirical results in visually realistic simulation environments show that the proposed model outperforms a wide range of ba…
Robotic navigation concerns the task in which a robot should be able to find a safe and feasible path and traverse between two points in a complex environment. We approach the problem of robotic navigation using reinforcement learning and use deep $Q$-networks to train agents to solve the task of robotic navigation. We compare the Entropy-Based Exploration (EBE) with the widely used $ε$-greedy exploration strategy by training agents using both of them in simulation. The trained agents are then tested on different versions of the environment to test the generalization ability of the learned pol…
In this paper, we present an autonomous navigation system for goal-driven exploration of unknown environments through deep reinforcement learning (DRL). Points of interest (POI) for possible navigation directions are obtained from the environment and an optimal waypoint is selected, based on the available data. Following the waypoints, the robot is guided towards the global goal and the local optimum problem of reactive navigation is mitigated. Then, a motion policy for local navigation is learned through a DRL framework in a simulation. We develop a navigation system where this learned policy…
University of Pennsylvania（宾夕法尼亚大学）；The University of Chicago（芝加哥大学）
We consider the problems of exploration and point-goal navigation in previously unseen environments, where the spatial complexity of indoor scenes and partial observability constitute these tasks challenging. We argue that learning occupancy priors over indoor maps provides significant advantages towards addressing these problems. To this end, we present a novel planning framework that first learns to generate occupancy maps beyond the field-of-view of the agent, and second leverages the model uncertainty over the generated areas to formulate path selection policies for each task of interest. …
AERR-Nav: Adaptive Exploration-Recovery-Reminiscing Strategy for Zero-Shot Object Navigation
Hong Kong Polytechnic University（香港理工大学）；Institute of automation, Chinese Academy of Sciences（中国科学院自动化研究所）；Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））
A new class of continuum robots has recently been explored, characterized by tip extension, significant length change, and directional control. Here, we call this class of robots "vine robots," due to their similar behavior to plants with the growth habit of trailing. Due to their growth-based movement, vine robots are well suited for navigation and exploration in cluttered environments, but until now, they have not been deployed outside the lab. Portability of these robots and steerability at length scales relevant for navigation are key to field applications. In addition, intuitive human-in-…
LERR-Nav: Learning to Coordinate Exploration, Recovery, and Reminiscing for Object Navigation
Hong Kong Polytechnic University（香港理工大学）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；DeepRobotics（深度机器人公司）；Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；Guangzhou University（广州大学）
Autonomous Exploration Development Environment is an open-source repository released to facilitate the development of high-level planning algorithms and integration of complete autonomous navigation systems. The repository contains representative simulation environment models, fundamental navigation modules, e.g., local planner, terrain traversability analysis, waypoint following, and visualization tools. Together with two of our high-level planner releases -- TARE planner for exploration and FAR planner for route planning, we detail usage of the three open-source repositories and share experi…
We present a strong baseline that surpasses the performance of previously published methods on the Habitat Challenge task of navigating to a target object in indoor environments. Our method is motivated from primary failure modes of prior state-of-the-art: poor exploration, inaccurate object identification, and agent getting trapped due to imprecise map construction. We make three contributions to mitigate these issues: (i) First, we show that existing map-based methods fail to effectively use semantic clues for exploration. We present a semantic-agnostic exploration strategy (called Stubborn)…
Building intelligent autonomous systems at any scale is challenging. The sensing and computation constraints of a microrobot platform make the problems harder. We present improvements to learning-based methods for on-board learning of locomotion, classification, and navigation of microrobots. We show how simulated locomotion can be achieved with model-based reinforcement learning via on-board sensor data distilled into control. Next, we introduce a sparse, linear detector and a Dynamic Thresholding method to FAST Visual Odometry for improved navigation in the noisy regime of mm scale imagery. …
Technische Universität München（慕尼黑工业大学）；Oculus Research（奥库鲁斯研究院）；RWTH Aachen University（亚琛工业大学）
Micro aerial vehicles (MAVs) are strongly limited in their payload and power capacity. In order to implement autonomous navigation, algorithms are therefore desirable that use sensory equipment that is as small, low-weight, and low-power consuming as possible. In this paper, we propose a method for autonomous MAV navigation and exploration using a low-cost consumer-grade quadrocopter equipped with a monocular camera. Our vision-based navigation system builds on LSD-SLAM which estimates the MAV trajectory and a semi-dense reconstruction of the environment in real-time. Since LSD-SLAM only deter…
Collaborative Navigation and Exploration with $β$-Sparse Gaussian Processes
D. Guggenheim School of Aerospace Engineering, Georgia Tech, Atlanta, GA, USA（佐治亚理工学院D.Guggenheim航空航天工程学院）；Department of Electrical and Computer Engineering, UNC Charlotte, Charlotte, NC, USA（北卡罗来纳大学夏洛特分校电气与计算机工程系）
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；University of Chinese Academy of Sciences（中国科学院大学）
This paper investigates the automatic exploration problem under the unknown environment, which is the key point of applying the robotic system to some social tasks. The solution to this problem via stacking decision rules is impossible to cover various environments and sensor properties. Learning based control methods are adaptive for these scenarios. However, these methods are damaged by low learning efficiency and awkward transferability from simulation to reality. In this paper, we construct a general exploration framework via decomposing the exploration process into the decision, planning,…
Nanjing University（南京大学）；University of New South Wales（新南威尔士大学）
For real-world deployments, it is critical to allow robots to navigate in complex environments autonomously. Traditional methods usually maintain an internal map of the environment, and then design several simple rules, in conjunction with a localization and planning approach, to navigate through the internal map. These approaches often involve a variety of assumptions and prior knowledge. In contrast, recent reinforcement learning (RL) methods can provide a model-free, self-learning mechanism as the robot interacts with an initially unknown environment, but are expensive to deploy in real-wor…
This study presents a new methodology for learning-based motion planning for autonomous exploration using aerial robots. Through the reinforcement learning method of learning through trial and error, the action policy is derived that can guide autonomous exploration of underground and tunnel environments. A new Markov decision process state is designed to learn the robot's action policy by using simulation only, and the results are applied to the real-world environment without further learning. Reduce the need for the precision map in grid-based path planner and achieve map-less navigation. Th…
Lookahead Exploration with Neural Radiance Representation for Continuous Vision-Language Navigation
Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）；University of Chinese Academy of Sciences（中国科学院大学）；University of Wisconsin（威斯康星大学）；Indiana University（印第安纳大学）
STEGNav: Spatio-Temporal Event Graph Reasoning for Multimodal Lifelong Object Navigation
State Key Laboratory of Novel Software Technology, Nanjing University（南京大学计算机软件新技术国家重点实验室）；School of Intelligence Science and Technology, Nanjing University（南京大学智能科学与技术学院）；School of Artificial Intelligence, Nanjing University（南京大学人工智能学院）
