This work presents a modular and hierarchical approach to learn policies for exploring 3D environments, called `Active Neural SLAM'. Our approach leverages the strengths of both classical and learning-based methods, by using analytical path planners with learned SLAM module, and global and local policies. The use of learning provides flexibility with respect to input modalities (in the SLAM module), leverages structural regularities of the world (in global policies), and provides robustness to errors in state estimation (in local policies). Such use of learning within each module retains its benefits, while at the same time, hierarchical decomposition and modular training allow us to sidestep the high sample complexities associated with training end-to-end policies. Our experiments in visually and physically realistic simulated 3D environments demonstrate the effectiveness of our approach over past learning and geometry-based approaches. The proposed model can also be easily transferred to the PointGoal task and was the winning entry of the CVPR 2019 Habitat PointGoal Navigation Challenge.
Dream-SLAM: Dreaming the Unseen for Active SLAM in Dynamic Environments
Thrust of Robotics and Autonomous Systems, Hong Kong University of Science and Technology (Guangzhou)（机器人与自主系统研究所，香港科学与技术大学（广州））；University of Zaragoza（阿拉瓦大学）；School of Computation, Information and Technology, Technical University of Munich（计算、信息与技术学院，慕尼黑技术大学）；School of Automation and IntelligentSensing, Shanghai Jiao Tong University（自动化与智能感知学院，上海交通大学）
Deploying autonomous robots capable of exploring unknown environments has long been a topic of great relevance to the robotics community. In this work, we take a further step in that direction by presenting an open-source active visual SLAM framework that leverages the accuracy of a state-of-the-art graph-SLAM system and takes advantage of the fast utility computation that exploiting the structure of the underlying pose-graph offers. Through careful estimation of a posteriori weighted pose-graphs, D-optimal decision-making is achieved online with the objective of improving localization and map…
Instituto de Investigación en Ingeniería de Aragón (I3A)（阿拉贡工程研究院（I3A））；Universidad de Zaragoza（萨拉戈萨大学）
Quantifying uncertainty is a key stage in active simultaneous localization and mapping (SLAM), as it allows to identify the most informative actions to execute. However, dealing with full covariance or even Fisher information matrices (FIMs) is computationally heavy and easily becomes intractable for online systems. In this work, we study the paradigm of active graph-SLAM formulated over \textit{SE(n)}, and propose a general relationship between the FIM of the system and the Laplacian matrix of the underlying pose-graph. This link makes possible to use graph connectivity indices as utility fun…
A Novel ViDAR Device With Visual Inertial Encoder Odometry and Reinforcement Learning-Based Active SLAM Method
State Key Laboratory for Turbulence and Complex Systems, Department of Advanced Manufacturing and Robotics, College of Engineering, Peking University（turbulence和复杂系统国家重点实验室，先进制造与机器人系，工程学院，北京大学）；Tencent RoboticsX（腾讯机器人科技）；College of Information and Electrical Engineering, China Agricultural University（信息与电气工程学院，中国农业大学）
Laboratoire des Sciences du Numérique de Nantes (LS2N)（南特数字科学实验室（LS2N））；CNRS（法国国家科学研究中心）；Ecole Centrale de Nantes（南特中央理工学院）；Capgemini Engineering（凯捷工程）
Active SLAM is the task of actively planning robot paths while simultaneously building a map and localizing within. Existing work has focused on planning paths with occupancy grid maps, which do not scale well and suffer from long term drift. This work proposes a Topological Feature Graph (TFG) representation that scales well and develops an active SLAM algorithm with it. The TFG uses graphical models, which utilize independences between variables, and enables a unified quantification of exploration and exploitation gains with a single entropy metric. Hence, it facilitates a natural and princi…
University of Freiburg（弗赖堡大学）；The Hong Kong University of Science and Technology（香港科技大学）
We present an approach for agents to learn representations of a global map from sensor data, to aid their exploration in new environments. To achieve this, we embed procedures mimicking that of traditional Simultaneous Localization and Mapping (SLAM) into the soft attention based addressing of external memory architectures, in which the external memory acts as an internal representation of the environment. This structure encourages the evolution of SLAM-like behaviors inside a completely differentiable deep neural network. We show that this approach can help reinforcement learning agents to su…
Max Planck Institute for Intelligent Systems（马克斯·普朗克智能系统研究所）；University of Stuttgart（斯图加特大学）
In active Visual-SLAM (V-SLAM), a robot relies on the information retrieved by its cameras to control its own movements for autonomous mapping of the environment. Cameras are usually statically linked to the robot's body, limiting the extra degrees of freedom for visual information acquisition. In this work, we overcome the aforementioned problem by introducing and leveraging an independently rotating camera on the robot base. This enables us to continuously control the heading of the camera, obtaining the desired optimal orientation for active V-SLAM, without rotating the robot itself. Howeve…
MA-SLAM: Active SLAM in Large-Scale Unknown Environment using Map Aware Deep Reinforcement Learning
Sun Yat-sen University（中山大学）；South China University of Technology（华南理工大学）；Hong Kong University of Science and Technology（香港科学与技术大学）；City University of Hong Kong（香港城市大学）
This paper proposes a novel active Simultaneous Localization and Mapping (SLAM) method with continuous trajectory optimization over a stochastic robot dynamics model. The problem is formalized as a stochastic optimal control over the continuous robot kinematic model to minimize a cost function that involves the covariance matrix of the landmark states. We tackle the problem by separately obtaining an open-loop control sequence subject to deterministic dynamics by iterative Covariance Regulation (iCR) and a closed-loop feedback control under stochastic robot and covariance dynamics by Linear Qu…
Northeastern University（东北大学）；The Chinese University of Hong Kong（香港中文大学）；Ulster University（阿尔斯特大学）
This paper presents the first active object mapping framework for complex robotic manipulation and autonomous perception tasks. The framework is built on an object SLAM system integrated with a simultaneous multi-object pose estimation process that is optimized for robotic grasping. Aiming to reduce the observation uncertainty on target objects and increase their pose estimation accuracy, we also design an object-driven exploration strategy to guide the object mapping process, enabling autonomous mapping and high-level perception. Combining the mapping module and the exploration strategy, an a…
Massive Parallel Deep Reinforcement Learning for Active SLAM
Facultad de Ciencias Exactas y Naturales, Universidad de Buenos Aires（布宜诺斯艾利斯大学自然科学与精确科学学院）；Instituto Tecnológico de Aragón (ITA)（阿拉贡技术研究所）；University of Zaragoza（萨拉戈萨大学）；INTELYMEC, Centro de Investigaciones en Física e Ingeniería del Centro (CIFICEN), UNICEN-CICPBA-CONICET（INTELYMEC，中部物理与工程研究中心（CIFICEN），国立中部大学-布宜诺斯艾利斯省科学研究委员会-国家科学与技术研究委员会）
Max Planck Institute for Intelligent Systems（马克斯·普朗克智能系统研究所）；University of Stuttgart（斯图加特大学）
In this paper, we present an active visual SLAM approach for omnidirectional robots. The goal is to generate control commands that allow such a robot to simultaneously localize itself and map an unknown environment while maximizing the amount of information gained and consuming as low energy as possible. Leveraging the robot's independent translation and rotation control, we introduce a multi-layered approach for active V-SLAM. The top layer decides on informative goal locations and generates highly informative paths to them. The second and third layers actively re-plan and execute the path, e…
LiDAR odometry is one of the essential parts of LiDAR simultaneous localization and mapping (SLAM). However, existing LiDAR odometry tends to match a new scan simply iteratively with previous fixed-pose scans, gradually accumulating errors. Furthermore, as an effective joint optimization mechanism, bundle adjustment (BA) cannot be directly introduced into real-time odometry due to the intensive computation of large-scale global landmarks. Therefore, this letter designs a new strategy named a landmark map for bundle adjustment odometry (LMBAO) in LiDAR SLAM to solve these problems. First, BA-ba…
Simultaneous Localization and Mapping, commonly known as SLAM, has been an active research area in the field of Robotics over the past three decades. For solving the SLAM problem, every robot is equipped with either a single sensor or a combination of similar/different sensors. This paper attempts to review, discuss, evaluate and compare these sensors. Keeping an eye on future, this paper also assesses the characteristics of these sensors against factors critical to the long-term autonomy challenge.
Contact SLAM: An Active Tactile Exploration Policy Based on Physical Reasoning Utilized in Robotic Fine Blind Manipulation Tasks
Research Center for Intelligent Robotics（智能机器人研究中心）；National Key Laboratory of Aerospace Flight Dynamics（航空飞行动力学国家重点实验室）；Shaanxi Province Innovation Team of Intelligent Robotic Technology（陕西省智能机器人技术创新团队）；School of Astronautics, Northwestern Polytechnical University（航空学院，西北工业大学）
Beihang University（北京航空航天大学）；University of Toronto（多伦多大学）
Object SLAM introduces the concept of objects into Simultaneous Localization and Mapping (SLAM) and helps understand indoor scenes for mobile robots and object-level interactive applications. The state-of-art object SLAM systems face challenges such as partial observations, occlusions, unobservable problems, limiting the mapping accuracy and robustness. This paper proposes a novel monocular Semantic Object SLAM (SO-SLAM) system that addresses the introduction of object spatial constraints. We explore three representative spatial constraints, including scale proportional constraint, symmetrical…
In autonomous robot exploration, the frontier is the border in the world map between the explored space and unexplored space. The frontier plays an important role when deciding where in the environment the robots should go explore next. We examine a modular control system pipeline for autonomous exploration where a 2D graph SLAM algorithm based on occupancy grid submaps performs map building and localization. We provide an overview of the state of the art in frontier detection and the relevant SLAM concepts and propose a specialized frontier detection method which is efficiently constrained to…
Reliability of SLAM systems is considered one of the critical requirements in modern autonomous systems. This directed the efforts to developing many state-of-the-art systems, creating challenging datasets, and introducing rigorous metrics to measure SLAM performance. However, the link between datasets and performance in the robustness/resilience context has rarely been explored. In order to fill this void, characterization of the operating conditions of SLAM systems is essential in order to provide an environment for quantitative measurement of robustness and resilience. In this paper, we arg…
Recent achievements in depth prediction from a single RGB image have powered the new research area of combining convolutional neural networks (CNNs) with classical simultaneous localization and mapping (SLAM) algorithms. The depth prediction from a CNN provides a reasonable initial point in the optimization process in the traditional SLAM algorithms, while the SLAM algorithms further improve the CNN prediction online. However, most of the current CNN-SLAM approaches have only taken advantage of the depth prediction but not yet other products from a CNN. In this work, we explore the use of the …
Robotics Institute, HKUST（香港科技大学机器人研究所）；Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）
Modern LiDAR-SLAM (L-SLAM) systems have shown excellent results in large-scale, real-world scenarios. However, they commonly have a high latency due to the expensive data association and nonlinear optimization. This paper demonstrates that actively selecting a subset of features significantly improves both the accuracy and efficiency of an L-SLAM system. We formulate the feature selection as a combinatorial optimization problem under a cardinality constraint to preserve the information matrix's spectral attributes. The stochastic-greedy algorithm is applied to approximate the optimal results i…
SLAM has matured significantly over the past few years, and is beginning to appear in serious commercial products. While new SLAM systems are being proposed at every conference, evaluation is often restricted to qualitative visualizations or accuracy estimation against a ground truth. This is due to the lack of benchmarking methodologies which can holistically and quantitatively evaluate these systems. Further investigation at the level of individual kernels and parameter spaces of SLAM pipelines is non-existent, which is absolutely essential for systems research and integration. We extend the…
Korea Advanced Institute of Science and Technology (KAIST)（韩国科学技术院）
In this paper, a degeneracy avoidance method for a point and line based visual SLAM algorithm is proposed. Visual SLAM predominantly uses point features. However, point features lack robustness in low texture and illuminance variant environments. Therefore, line features are used to compensate the weaknesses of point features. In addition, point features are poor in representing discernable features for the naked eye, meaning mapped point features cannot be recognized. To overcome the limitations above, line features were actively employed in previous studies. However, since degeneracy arises …
Benchmarking Simultaneous Localization and Mapping (SLAM) algorithms is important to scientists and users of robotic systems alike. But through their many configuration options in hardware and software, SLAM systems feature a vast parameter space that scientists up to now were not able to explore. The proposed SLAM Hive Benchmarking Suite is able to analyze SLAM algorithms in 1000's of mapping runs, through its utilization of container technology and deployment in a cluster. This paper presents the architecture and open source implementation of SLAM Hive and compares it to existing efforts on …
Departamento de Informática e Ingeniería de Sistemas（信息与系统工程系）；Instituto de Investigación en Ingeniería de Aragón（阿拉贡工程研究所）；Universidad de Zaragoza（萨拉戈萨大学）
In this work, we fully define the existing relationships between traditional optimality criteria and the connectivity of the underlying pose-graph in Active SLAM, characterizing, therefore, the connection between Graph Theory and the Theory Optimal Experimental Design. We validate the proposed relationships in 2D and 3D graph SLAM datasets, showing a remarkable relaxation of the computational load when using the graph structure. Furthermore, we present a novel Active SLAM framework which outperforms traditional methods by successfully leveraging the graphical facet of the problem so as to auto…
DFKI - German Research Center for Artificial Intelligence（德国人工智能研究中心）
This paper demonstrates a system capable of combining a sparse, indirect, monocular visual SLAM, with both offline and real-time Multi-View Stereo (MVS) reconstruction algorithms. This combination overcomes many obstacles encountered by autonomous vehicles or robots employed in agricultural environments, such as overly repetitive patterns, need for very detailed reconstructions, and abrupt movements caused by uneven roads. Furthermore, the use of a monocular SLAM makes our system much easier to integrate with an existing device, as we do not rely on a LiDAR (which is expensive and power consum…
It holds great implications for practical applications to enable centimeter-accuracy positioning for mobile and wearable sensor systems. In this paper, we propose a novel, high-precision, efficient visual-inertial (VI)-SLAM algorithm, termed Schmidt-EKF VI-SLAM (SEVIS), which optimally fuses IMU measurements and monocular images in a tightly-coupled manner to provide 3D motion tracking with bounded error. In particular, we adapt the Schmidt Kalman filter formulation to selectively include informative features in the state vector while treating them as nuisance parameters (or Schmidt states) on…
This paper presents a hybrid real-time camera pose estimation framework with a novel partitioning scheme and introduces motion averaging to monocular Simultaneous Localization and Mapping (SLAM) systems. Breaking through the limitations of fixed-size temporal partitioning in many conventional SLAM pipelines, our approach significantly improves the accuracy of local bundle adjustment by gathering spatially-strongly-connected cameras into each block. With the dynamic initialization using intermediate computation values, \XL{we improve the Levenberg-Marquardt solver to further enhance the efficie…
School of System Design and Intelligent Manufacturing, Southern University of Science and Technology（南方科技大学系统设计与智能制造学院）
Combining lidar in camera-based simultaneous localization and mapping (SLAM) is an effective method in improving overall accuracy, especially at a large scale outdoor scenario. Recent development of low-cost lidars (e.g. Livox lidar) enable us to explore such SLAM systems with lower budget and higher performance. In this paper we propose CamVox by adapting Livox lidars into visual SLAM (ORB-SLAM2) by exploring the lidars' unique features. Based on the non-repeating nature of Livox lidars, we propose an automatic lidar-camera calibration method that will work in uncontrolled scenes. The long de…
Energy-efficient simultaneous localization and mapping (SLAM) is crucial for mobile robots exploring unknown environments. The mammalian brain solves SLAM via a network of specialized neurons, exhibiting asynchronous computations and event-based communications, with very low energy consumption. We propose a brain-inspired spiking neural network (SNN) architecture that solves the unidimensional SLAM by introducing spike-based reference frame transformation, visual likelihood computation, and Bayesian inference. We integrated our neuromorphic algorithm to Intel's Loihi neuromorphic processor, a …
MAGS-SLAM: Monocular Multi-Agent Gaussian Splatting SLAM for Geometrically and Photometrically Consistent Reconstruction
ETH Zurich（苏黎世联邦理工学院）；Harbin Engineering University（哈尔滨工程大学）；University of Liverpool（利物浦大学）；University of Macau（澳门大学）；University of Ottawa（多伦多大学）；Wuhan University（武汉大学）；Imperial College London（伦敦帝国学院）
Australian National University（澳大利亚国立大学）；Brno University of Technology（布尔诺理工大学）
Maximum likelihood estimation (MLE) is a well-known estimation method used in many robotic and computer vision applications. Under Gaussian assumption, the MLE converts to a nonlinear least squares (NLS) problem. Efficient solutions to NLS exist and they are based on iteratively solving sparse linear systems until convergence. In general, the existing solutions provide only an estimation of the mean state vector, the resulting covariance being computationally too expensive to recover. Nevertheless, in many simultaneous localisation and mapping (SLAM) applications, knowing only the mean vector …
