We present a fast, scalable, and accurate Simultaneous Localization and Mapping (SLAM) system that represents indoor scenes as a graph of objects. Leveraging the observation that artificial environments are structured and occupied by recognizable objects, we show that a compositional scalable object mapping formulation is amenable to a robust SLAM solution for drift-free large scale indoor reconstruction. To achieve this, we propose a novel semantically assisted data association strategy that obtains unambiguous persistent object landmarks, and a 2.5D compositional rendering method that enables reliable frame-to-model RGB-D tracking. Consequently, we deliver an optimized online implementation that can run at near frame rate with a single graphics card, and provide a comprehensive evaluation against state of the art baselines. An open source implementation will be provided at https://placeholder.
The current state of the art of Simultaneous Localisation and Mapping, or SLAM, on low power embedded systems is about sparse localisation and mapping with low resolution results in the name of efficiency. Meanwhile, research in this field has provided many advances for information rich processing and semantic understanding, combined with high computational requirements for real-time processing. This work provides a solution to bridging this gap, in the form of a scalable SLAM-specific architecture for depth estimation for direct semi-dense SLAM. Targeting an off-the-shelf FPGA-SoC this accele…
FusionPortableV2: A Unified Multi-Sensor Dataset for Generalized SLAM Across Diverse Platforms and Scalable Environments
The Hong Kong University of Science and Technology（香港科技大学）；University College London（伦敦大学学院）；Southern University of Science and Technology（南方科技大学）；The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））
Scalable Outdoors Autonomous Drone Flight with Visual-Inertial SLAM and Dense Submaps Built without LiDAR
Technical University of Munich（技术大学慕尼黑）；School of Computation, Information and Technology（计算、信息与技术学院）；Mobile Robotics Lab（移动机器人实验室）；Department of Computing（计算学院）；Imperial College London（伦敦帝国学院）；Department of Mechanical and Process Engineering（机械与过程工程学院）；ETH Zurich（苏黎世联邦理工学院）；Munich Institute of Robotics and Machine Intelligence (MIRMI)（慕尼黑机器人与机器智能研究所）；Munich Center for Machine Learning (MCML)（慕尼黑机器学习中心）
University of Zurich（苏黎世大学）；ETH Zurich（苏黎世联邦理工学院）；Sony R&D Center Europe（索尼欧洲研发中心）
Adding more cameras to SLAM systems improves robustness and accuracy but complicates the design of the visual front-end significantly. Thus, most systems in the literature are tailored for specific camera configurations. In this work, we aim at an adaptive SLAM system that works for arbitrary multi-camera setups. To this end, we revisit several common building blocks in visual SLAM. In particular, we propose an adaptive initialization scheme, a sensor-agnostic, information-theoretic keyframe selection algorithm, and a scalable voxel-based map. These techniques make little assumption about the …
In this work, we develop a monocular SLAM-aware object recognition system that is able to achieve considerably stronger recognition performance, as compared to classical object recognition systems that function on a frame-by-frame basis. By incorporating several key ideas including multi-view object proposals and efficient feature encoding methods, our proposed system is able to detect and robustly recognize objects in its environment using a single RGB camera in near-constant time. Through experiments, we illustrate the utility of using such a system to effectively detect and recognize object…
Massive Parallel Deep Reinforcement Learning for Active SLAM
Facultad de Ciencias Exactas y Naturales, Universidad de Buenos Aires（布宜诺斯艾利斯大学自然科学与精确科学学院）；Instituto Tecnológico de Aragón (ITA)（阿拉贡技术研究所）；University of Zaragoza（萨拉戈萨大学）；INTELYMEC, Centro de Investigaciones en Física e Ingeniería del Centro (CIFICEN), UNICEN-CICPBA-CONICET（INTELYMEC，中部物理与工程研究中心（CIFICEN），国立中部大学-布宜诺斯艾利斯省科学研究委员会-国家科学与技术研究委员会）
This paper introduces SD-6DoF-ICLK, a learning-based Inverse Compositional Lucas-Kanade (ICLK) pipeline that uses sparse depth information to optimize the relative pose that best aligns two images on SE(3). To compute this six Degrees-of-Freedom (DoF) relative transformation, the proposed formulation requires only sparse depth information in one of the images, which is often the only available depth source in visual-inertial odometry or Simultaneous Localization and Mapping (SLAM) pipelines. In an optional subsequent step, the framework further refines feature locations and the relative pose u…
Collaborative SLAM enables a group of agents to simultaneously co-localize and jointly map an environment, thus paving the way to wide-ranging applications of multi-robot perception and multi-user AR experiences by eliminating the need for external infrastructure or pre-built maps. This article presents COVINS, a novel collaborative SLAM system, that enables multi-agent, scalable SLAM in large environments and for large teams of more than 10 agents. The paradigm here is that each agent runs visual-inertial odomety independently onboard in order to ensure its autonomy, while sharing map informa…
GLAM-SLAM: Real-time Gaussian Large-scale Mapping via Flow Densification and Spatial Decomposition
HERON - Hellenic Robotics Center of Excellence, Athena Research Center（希腊机器人卓越中心 - 阿西娜研究中心）；Institute of Robotics, Athena Research Center（阿西娜研究中心机器人研究所）；National Technical University of Athens（雅典国立技术大学）
Massachusetts Institute of Technology（麻省理工学院）；Woods Hole Oceanographic Institution（伍兹霍尔海洋研究所）；Northeastern University（东北大学）
We present a novel initialization technique for the range-aided simultaneous localization and mapping (RA-SLAM) problem. In RA-SLAM we consider measurements of point-to-point distances in addition to measurements of rigid transformations to landmark or pose variables. Standard formulations of RA-SLAM approach the problem as non-convex optimization, which requires a good initialization to obtain quality results. The initialization technique proposed here relaxes the RA-SLAM problem to a convex problem which is then solved to determine an initialization for the original, non-convex problem. The …
The Hong Kong University of Science and Technology（香港科技大学）
In this paper, we propose a novel dense surfel mapping system that scales well in different environments with only CPU computation. Using a sparse SLAM system to estimate camera poses, the proposed mapping system can fuse intensity images and depth images into a globally consistent model. The system is carefully designed so that it can build from room-scale environments to urban-scale environments using depth images from RGB-D cameras, stereo cameras or even a monocular camera. First, superpixels extracted from both intensity and depth images are used to model surfels in the system. superpixel…
Australian National University（澳大利亚国立大学）；Brno University of Technology（布尔诺理工大学）
Maximum likelihood estimation (MLE) is a well-known estimation method used in many robotic and computer vision applications. Under Gaussian assumption, the MLE converts to a nonlinear least squares (NLS) problem. Efficient solutions to NLS exist and they are based on iteratively solving sparse linear systems until convergence. In general, the existing solutions provide only an estimation of the mean state vector, the resulting covariance being computationally too expensive to recover. Nevertheless, in many simultaneous localisation and mapping (SLAM) applications, knowing only the mean vector …
3D Foundation Model-Based Loop Closing for Decentralized Collaborative SLAM
Department of Computer and Software Engineering, Polytechnique Montréal（计算机与软件工程系，蒙特利尔理工学院）；Oxford Robotics Institute, Dept. of Engineering Science, University of Oxford（牛津大学机器人研究所，工程科学系）
Beijing University of Posts and Telecommunications（北京邮电大学）；Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）
State estimation with sensors is essential for mobile robots. Due to different performance of sensors in different environments, how to fuse measurements of various sensors is a problem. In this paper, we propose a tightly coupled multi-sensor fusion framework, Lvio-Fusion, which fuses stereo camera, Lidar, IMU, and GPS based on the graph optimization. Especially for urban traffic scenes, we introduce a segmented global pose graph optimization with GPS and loop-closure, which can eliminate accumulated drifts. Additionally, we creatively use a actor-critic method in reinforcement learning to ad…
We propose an efficient and scalable method for incrementally building a dense, semantically annotated 3D map in real-time. The proposed method assigns class probabilities to each region, not each element (e.g., surfel and voxel), of the 3D map which is built up through a robust SLAM framework and incrementally segmented with a geometric-based segmentation method. Differently from all other approaches, our method has a capability of running at over 30Hz while performing all processing components, including SLAM, segmentation, 2D recognition, and updating class probabilities of each segmentatio…
Breaking Time: A Fully Gaussian Framework for Distributed and Continuous-Time SLAM
Department of Computer, Control, and Management Engineering "Antonio Ruberti", Sapienza University of Rome（计算机、控制与管理工程系（Antonio Ruberti）, 罗马萨皮恩扎大学）；University of Stuttgart（斯图加特大学）
MDF: Multi-Modal Data Fusion with CNN-Based Object Detection for Enhanced Indoor Localization Using LiDAR-SLAM
Department of Artificial Intelligence Convergence, Pukyong National University（人工智能融合系，浦项国立大学）；School of Computer Science, University of Nottingham Ningbo China（计算机科学学院，诺丁汉国际大学）
OKVIS2-X: Open Keyframe-based Visual-Inertial SLAM Configurable with Dense Depth or LiDAR, and GNSS
Mobile Robotics Lab, School of Computation, Information and Technology (CIT) at the Technical University of Munich (TUM)（慕尼黑技术大学移动机器人实验室，计算信息与技术学院）；Munich Institute of Robotics and Machine Intelligence (MIRMI)（慕尼黑机器人与机器智能研究所）；Mobile Robotics Lab, ETH Zurich（苏黎世联邦理工学院移动机器人实验室）
National Institute of Advanced Industrial Science and Technology（产业技术综合研究所）
This paper presents an accurate and scalable method for fiducial tag localization on a 3D prior environmental map. The proposed method comprises three steps: 1) visual odometry-based landmark SLAM for estimating the relative poses between fiducial tags, 2) geometrical matching-based global tag-map registration via maximum clique finding, and 3) tag pose refinement based on direct camera-map alignment with normalized information distance. Through simulation-based evaluations, the proposed method achieved a 98 \% global tag-map registration success rate and an average tag pose estimation accurac…
Massachusetts Institute of Technology（麻省理工学院）；NASA Jet Propulsion Laboratory – California Institute of Technology（美国国家航空航天局喷气推进实验室——加州理工学院）；University of Southern California（南加州大学）；University of Bonn（波恩大学）；Polytechnic University of Bari（巴里理工大学）；Bangladesh University of Engineering and Technology（孟加拉国工程技术大学）
Search and rescue with a team of heterogeneous mobile robots in unknown and large-scale underground environments requires high-precision localization and mapping. This crucial requirement is faced with many challenges in complex and perceptually-degraded subterranean environments, as the onboard perception system is required to operate in off-nominal conditions (poor visibility due to darkness and dust, rugged and muddy terrain, and the presence of self-similar and ambiguous scenes). In a disaster response scenario and in the absence of prior information about the environment, robots must rely…
Adaptive World Memory 3D Foundation Model for Scalable 3D Mapping, Localization, and Rendering
Shanghai Jiao Tong University（上海交通大学）；State Key Laboratory of Avionics Integration and Aviation System-of-Systems Synthesis（航空电子综合与航空系统体系综合国家重点实验室）；Shanghai Key Laboratory of Navigation and Location Based Services（上海市导航与位置服务重点实验室）；Nanyang Technological University（南洋理工大学）；University of Technology Nuremberg（纽伦堡工业大学）
Ritsumeikan University（立命馆大学）；National Institute of Informatics（国立信息学研究所）；SOKENDAI (The Graduate University for Advanced Studies)（综合研究大学院大学）
We propose a novel online learning algorithm, called SpCoSLAM 2.0, for spatial concepts and lexical acquisition with high accuracy and scalability. Previously, we proposed SpCoSLAM as an online learning algorithm based on unsupervised Bayesian probabilistic model that integrates multimodal place categorization, lexical acquisition, and SLAM. However, our original algorithm had limited estimation accuracy owing to the influence of the early stages of learning, and increased computational complexity with added training data. Therefore, we introduce techniques such as fixed-lag rejuvenation to re…
Jet Propulsion Laboratory - California Institute of Technology（加州理工学院喷气推进实验室）；National Aeronautics and Space Administration（美国国家航空航天局）；Defense Advanced Research Projects Agency（美国国防高级研究计划局）
Multi-robot SLAM systems in GPS-denied environments require loop closures to maintain a drift-free centralized map. With an increasing number of robots and size of the environment, checking and computing the transformation for all the loop closure candidates becomes computationally infeasible. In this work, we describe a loop closure module that is able to prioritize which loop closures to compute based on the underlying pose graph, the proximity to known beacons, and the characteristics of the point clouds. We validate this system in the context of the DARPA Subterranean Challenge and on nume…
ComSim: Building Scalable Real-World Robot Data Generation via Compositional Simulation
CUHK-Shenzhen（香港中文大学（深圳））；Sun Yat-sen University（中山大学）；Shanghai Jiao Tong University（上海交通大学）；USTC（中国科学技术大学）；The University of Hong Kong（香港大学）；University of Oxford（牛津大学）；Harvard University（哈佛大学）
ASSIST: Interactive Scene Nodes for Scalable and Realistic Indoor Simulation
Institute for AI Industry Research (AIR), Tsinghua University（清华大学智能产业研究院（AIR））；Institute of Software, Chinese Academy of Sciences（中国科学院软件研究所）；Beijing University of Civil Engineering and Architecture（北京建筑大学）；University of Science and Technology of China（中国科学技术大学）；McGill University（麦吉尔大学）；Beijing Institute（北京研究所）
