We present X-SLAM, a real-time dense differentiable SLAM system that leverages the complex-step finite difference (CSFD) method for efficient calculation of numerical derivatives, bypassing the need for a large-scale computational graph. The key to our approach is treating the SLAM process as a differentiable function, enabling the calculation of the derivatives of important SLAM parameters through Taylor series expansion within the complex domain. Our system allows for the real-time calculation of not just the gradient, but also higher-order differentiation. This facilitates the use of high-order optimizers to achieve better accuracy and faster convergence. Building on X-SLAM, we implemented end-to-end optimization frameworks for two important tasks: camera relocalization in wide outdoor scenes and active robotic scanning in complex indoor environments. Comprehensive evaluations on public benchmarks and intricate real scenes underscore the improvements in the accuracy of camera relocalization and the efficiency of robotic navigation achieved through our task-aware optimization. The code and data are available at https://gapszju.github.io/X-SLAM.
The current state of the art of Simultaneous Localisation and Mapping, or SLAM, on low power embedded systems is about sparse localisation and mapping with low resolution results in the name of efficiency. Meanwhile, research in this field has provided many advances for information rich processing and semantic understanding, combined with high computational requirements for real-time processing. This work provides a solution to bridging this gap, in the form of a scalable SLAM-specific architecture for depth estimation for direct semi-dense SLAM. Targeting an off-the-shelf FPGA-SoC this accele…
We present a fast, scalable, and accurate Simultaneous Localization and Mapping (SLAM) system that represents indoor scenes as a graph of objects. Leveraging the observation that artificial environments are structured and occupied by recognizable objects, we show that a compositional scalable object mapping formulation is amenable to a robust SLAM solution for drift-free large scale indoor reconstruction. To achieve this, we propose a novel semantically assisted data association strategy that obtains unambiguous persistent object landmarks, and a 2.5D compositional rendering method that enable…
FusionPortableV2: A Unified Multi-Sensor Dataset for Generalized SLAM Across Diverse Platforms and Scalable Environments
The Hong Kong University of Science and Technology（香港科技大学）；University College London（伦敦大学学院）；Southern University of Science and Technology（南方科技大学）；The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））
Scalable Outdoors Autonomous Drone Flight with Visual-Inertial SLAM and Dense Submaps Built without LiDAR
Technical University of Munich（技术大学慕尼黑）；School of Computation, Information and Technology（计算、信息与技术学院）；Mobile Robotics Lab（移动机器人实验室）；Department of Computing（计算学院）；Imperial College London（伦敦帝国学院）；Department of Mechanical and Process Engineering（机械与过程工程学院）；ETH Zurich（苏黎世联邦理工学院）；Munich Institute of Robotics and Machine Intelligence (MIRMI)（慕尼黑机器人与机器智能研究所）；Munich Center for Machine Learning (MCML)（慕尼黑机器学习中心）
Massive Parallel Deep Reinforcement Learning for Active SLAM
Facultad de Ciencias Exactas y Naturales, Universidad de Buenos Aires（布宜诺斯艾利斯大学自然科学与精确科学学院）；Instituto Tecnológico de Aragón (ITA)（阿拉贡技术研究所）；University of Zaragoza（萨拉戈萨大学）；INTELYMEC, Centro de Investigaciones en Física e Ingeniería del Centro (CIFICEN), UNICEN-CICPBA-CONICET（INTELYMEC，中部物理与工程研究中心（CIFICEN），国立中部大学-布宜诺斯艾利斯省科学研究委员会-国家科学与技术研究委员会）
Massachusetts Institute of Technology（麻省理工学院）；Woods Hole Oceanographic Institution（伍兹霍尔海洋研究所）；Northeastern University（东北大学）
We present a novel initialization technique for the range-aided simultaneous localization and mapping (RA-SLAM) problem. In RA-SLAM we consider measurements of point-to-point distances in addition to measurements of rigid transformations to landmark or pose variables. Standard formulations of RA-SLAM approach the problem as non-convex optimization, which requires a good initialization to obtain quality results. The initialization technique proposed here relaxes the RA-SLAM problem to a convex problem which is then solved to determine an initialization for the original, non-convex problem. The …
Australian National University（澳大利亚国立大学）；Brno University of Technology（布尔诺理工大学）
Maximum likelihood estimation (MLE) is a well-known estimation method used in many robotic and computer vision applications. Under Gaussian assumption, the MLE converts to a nonlinear least squares (NLS) problem. Efficient solutions to NLS exist and they are based on iteratively solving sparse linear systems until convergence. In general, the existing solutions provide only an estimation of the mean state vector, the resulting covariance being computationally too expensive to recover. Nevertheless, in many simultaneous localisation and mapping (SLAM) applications, knowing only the mean vector …
3D Foundation Model-Based Loop Closing for Decentralized Collaborative SLAM
Department of Computer and Software Engineering, Polytechnique Montréal（计算机与软件工程系，蒙特利尔理工学院）；Oxford Robotics Institute, Dept. of Engineering Science, University of Oxford（牛津大学机器人研究所，工程科学系）
University of Zurich（苏黎世大学）；ETH Zurich（苏黎世联邦理工学院）；Sony R&D Center Europe（索尼欧洲研发中心）
Adding more cameras to SLAM systems improves robustness and accuracy but complicates the design of the visual front-end significantly. Thus, most systems in the literature are tailored for specific camera configurations. In this work, we aim at an adaptive SLAM system that works for arbitrary multi-camera setups. To this end, we revisit several common building blocks in visual SLAM. In particular, we propose an adaptive initialization scheme, a sensor-agnostic, information-theoretic keyframe selection algorithm, and a scalable voxel-based map. These techniques make little assumption about the …
Beijing University of Posts and Telecommunications（北京邮电大学）；Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）
State estimation with sensors is essential for mobile robots. Due to different performance of sensors in different environments, how to fuse measurements of various sensors is a problem. In this paper, we propose a tightly coupled multi-sensor fusion framework, Lvio-Fusion, which fuses stereo camera, Lidar, IMU, and GPS based on the graph optimization. Especially for urban traffic scenes, we introduce a segmented global pose graph optimization with GPS and loop-closure, which can eliminate accumulated drifts. Additionally, we creatively use a actor-critic method in reinforcement learning to ad…
In this work, we develop a monocular SLAM-aware object recognition system that is able to achieve considerably stronger recognition performance, as compared to classical object recognition systems that function on a frame-by-frame basis. By incorporating several key ideas including multi-view object proposals and efficient feature encoding methods, our proposed system is able to detect and robustly recognize objects in its environment using a single RGB camera in near-constant time. Through experiments, we illustrate the utility of using such a system to effectively detect and recognize object…
Breaking Time: A Fully Gaussian Framework for Distributed and Continuous-Time SLAM
Department of Computer, Control, and Management Engineering "Antonio Ruberti", Sapienza University of Rome（计算机、控制与管理工程系（Antonio Ruberti）, 罗马萨皮恩扎大学）；University of Stuttgart（斯图加特大学）
Collaborative SLAM enables a group of agents to simultaneously co-localize and jointly map an environment, thus paving the way to wide-ranging applications of multi-robot perception and multi-user AR experiences by eliminating the need for external infrastructure or pre-built maps. This article presents COVINS, a novel collaborative SLAM system, that enables multi-agent, scalable SLAM in large environments and for large teams of more than 10 agents. The paradigm here is that each agent runs visual-inertial odomety independently onboard in order to ensure its autonomy, while sharing map informa…
MDF: Multi-Modal Data Fusion with CNN-Based Object Detection for Enhanced Indoor Localization Using LiDAR-SLAM
Department of Artificial Intelligence Convergence, Pukyong National University（人工智能融合系，浦项国立大学）；School of Computer Science, University of Nottingham Ningbo China（计算机科学学院，诺丁汉国际大学）
OKVIS2-X: Open Keyframe-based Visual-Inertial SLAM Configurable with Dense Depth or LiDAR, and GNSS
Mobile Robotics Lab, School of Computation, Information and Technology (CIT) at the Technical University of Munich (TUM)（慕尼黑技术大学移动机器人实验室，计算信息与技术学院）；Munich Institute of Robotics and Machine Intelligence (MIRMI)（慕尼黑机器人与机器智能研究所）；Mobile Robotics Lab, ETH Zurich（苏黎世联邦理工学院移动机器人实验室）
National Institute of Advanced Industrial Science and Technology（产业技术综合研究所）
This paper presents an accurate and scalable method for fiducial tag localization on a 3D prior environmental map. The proposed method comprises three steps: 1) visual odometry-based landmark SLAM for estimating the relative poses between fiducial tags, 2) geometrical matching-based global tag-map registration via maximum clique finding, and 3) tag pose refinement based on direct camera-map alignment with normalized information distance. Through simulation-based evaluations, the proposed method achieved a 98 \% global tag-map registration success rate and an average tag pose estimation accurac…
GLAM-SLAM: Real-time Gaussian Large-scale Mapping via Flow Densification and Spatial Decomposition
HERON - Hellenic Robotics Center of Excellence, Athena Research Center（希腊机器人卓越中心 - 阿西娜研究中心）；Institute of Robotics, Athena Research Center（阿西娜研究中心机器人研究所）；National Technical University of Athens（雅典国立技术大学）
The Hong Kong University of Science and Technology（香港科技大学）
In this paper, we propose a novel dense surfel mapping system that scales well in different environments with only CPU computation. Using a sparse SLAM system to estimate camera poses, the proposed mapping system can fuse intensity images and depth images into a globally consistent model. The system is carefully designed so that it can build from room-scale environments to urban-scale environments using depth images from RGB-D cameras, stereo cameras or even a monocular camera. First, superpixels extracted from both intensity and depth images are used to model surfels in the system. superpixel…
Massachusetts Institute of Technology（麻省理工学院）；NASA Jet Propulsion Laboratory – California Institute of Technology（美国国家航空航天局喷气推进实验室——加州理工学院）；University of Southern California（南加州大学）；University of Bonn（波恩大学）；Polytechnic University of Bari（巴里理工大学）；Bangladesh University of Engineering and Technology（孟加拉国工程技术大学）
Search and rescue with a team of heterogeneous mobile robots in unknown and large-scale underground environments requires high-precision localization and mapping. This crucial requirement is faced with many challenges in complex and perceptually-degraded subterranean environments, as the onboard perception system is required to operate in off-nominal conditions (poor visibility due to darkness and dust, rugged and muddy terrain, and the presence of self-similar and ambiguous scenes). In a disaster response scenario and in the absence of prior information about the environment, robots must rely…
We propose an efficient and scalable method for incrementally building a dense, semantically annotated 3D map in real-time. The proposed method assigns class probabilities to each region, not each element (e.g., surfel and voxel), of the 3D map which is built up through a robust SLAM framework and incrementally segmented with a geometric-based segmentation method. Differently from all other approaches, our method has a capability of running at over 30Hz while performing all processing components, including SLAM, segmentation, 2D recognition, and updating class probabilities of each segmentatio…
Jet Propulsion Laboratory - California Institute of Technology（加州理工学院喷气推进实验室）；National Aeronautics and Space Administration（美国国家航空航天局）；Defense Advanced Research Projects Agency（美国国防高级研究计划局）
Multi-robot SLAM systems in GPS-denied environments require loop closures to maintain a drift-free centralized map. With an increasing number of robots and size of the environment, checking and computing the transformation for all the loop closure candidates becomes computationally infeasible. In this work, we describe a loop closure module that is able to prioritize which loop closures to compute based on the underlying pose graph, the proximity to known beacons, and the characteristics of the point clouds. We validate this system in the context of the DARPA Subterranean Challenge and on nume…
Ritsumeikan University（立命馆大学）；National Institute of Informatics（国立信息学研究所）；SOKENDAI (The Graduate University for Advanced Studies)（综合研究大学院大学）
We propose a novel online learning algorithm, called SpCoSLAM 2.0, for spatial concepts and lexical acquisition with high accuracy and scalability. Previously, we proposed SpCoSLAM as an online learning algorithm based on unsupervised Bayesian probabilistic model that integrates multimodal place categorization, lexical acquisition, and SLAM. However, our original algorithm had limited estimation accuracy owing to the influence of the early stages of learning, and increased computational complexity with added training data. Therefore, we introduce techniques such as fixed-lag rejuvenation to re…
Adaptive World Memory 3D Foundation Model for Scalable 3D Mapping, Localization, and Rendering
Shanghai Jiao Tong University（上海交通大学）；State Key Laboratory of Avionics Integration and Aviation System-of-Systems Synthesis（航空电子综合与航空系统体系综合国家重点实验室）；Shanghai Key Laboratory of Navigation and Location Based Services（上海市导航与位置服务重点实验室）；Nanyang Technological University（南洋理工大学）；University of Technology Nuremberg（纽伦堡工业大学）
DERD-Net: Learning Depth from Event-based Ray Densities
Technische Universität Berlin（柏林技术大学）；Einstein Center Digital Future（爱因斯坦数字未来中心）；Robotics Institute（机器人研究所）；Science of Intelligence Excellence Cluster（智能科学卓越集群）
General Place Recognition Survey: Towards Real-World Autonomy
City University of Hong Kong（香港城市大学）；University College London（伦敦大学学院）；University of Delaware（特拉华大学）；Carnegie Mellon University（卡内基梅隆大学）；Nankai University（南开大学）
