Combining Simultaneous Localisation and Mapping (SLAM) estimation and dynamic scene modelling can highly benefit robot autonomy in dynamic environments. Robot path planning and obstacle avoidance tasks rely on accurate estimations of the motion of dynamic objects in the scene. This paper presents VDO-SLAM, a robust visual dynamic object-aware SLAM system that exploits semantic information to enable accurate motion estimation and tracking of dynamic rigid objects in the scene without any prior knowledge of the objects' shape or geometric models. The proposed approach identifies and tracks the dynamic objects and the static structure in the environment and integrates this information into a unified SLAM framework. This results in highly accurate estimates of the robot's trajectory and the full SE(3) motion of the objects as well as a spatiotemporal map of the environment. The system is able to extract linear velocity estimates from objects' SE(3) motion providing an important functionality for navigation in complex dynamic environments. We demonstrate the performance of the proposed system on a number of real indoor and outdoor datasets and the results show consistent and substantial improvements over the state-of-the-art algorithms. An open-source version of the source code is available.
Simultaneous Localization & Mapping (SLAM) is the process of building a mutual relationship between localization and mapping of the subject in its surrounding environment. With the help of different sensors, various types of SLAM systems have developed to deal with the problem of building the relationship between localization and mapping. A limitation in the SLAM process is the lack of consideration of dynamic objects in the mapping of the environment. We propose the Dynamic Object Tracking SLAM (DyOb-SLAM), which is a Visual SLAM system that can localize and map the surrounding dynamic object…
DAGS-SLAM: Dynamic-Aware 3DGS SLAM via Spatiotemporal Motion Probability and Uncertainty-Aware Scheduling
School of Mathematics, Hefei University of Technology（合肥工业大学数学学院）；School of Software, Tsinghua University（清华大学软件学院）；School of Computer Science and Technology, University of Science and Technology of China（中国科学技术大学计算机科学与技术学院）
POV-SLAM: Probabilistic Object-Aware Variational SLAM in Semi-Static Environments
The University of Toronto Institute for Aerospace Studies（多伦多大学航空航天研究所）；The Technical University of Munich（慕尼黑工业大学）；Clearpath Robotics（Clearpath机器人公司）；The Technical University of Nuremberg（纽伦堡工业大学）
Massachusetts Institute of Technology (MIT)（麻省理工学院）
Building object-level maps can facilitate robot-environment interactions (e.g. planning and manipulation), but objects could often have multiple probable poses when viewed from a single vantage point, due to symmetry, occlusion or perceptual failures. A robust object-level simultaneous localization and mapping (object SLAM) algorithm needs to be aware of this pose ambiguity. We propose to maintain and subsequently disambiguate the multiple pose interpretations to gradually recover a globally consistent world representation. The max-mixtures model is applied to implicitly and efficiently track …
DYNEMO-SLAM: Dynamic Entity and Motion-Aware 3D Scene Graph SLAM
Automation and Robotics Research Group, Interdisciplinary Centre for Security, Reliability and Trust (SnT), University of Luxembourg（卢森堡大学自动化与机器人研究组，安全、可靠与信任跨学科研究中心（SnT））
Delivery robots aim to achieve high precision to facilitate complete autonomy. A precise three-dimensional point cloud map of sidewalk surroundings is required to estimate self-location. With or without the loop closing method, the cumulative error increases gradually after mapping for larger urban or city maps due to sensor drift. Therefore, there is a high risk of using the drifted or misaligned map. This article presented a technique for fusing GPS to update the 3D point cloud and eliminate cumulative error. The proposed method shows outstanding results in quantitative comparison and qualit…
MA-SLAM: Active SLAM in Large-Scale Unknown Environment using Map Aware Deep Reinforcement Learning
Sun Yat-sen University（中山大学）；South China University of Technology（华南理工大学）；Hong Kong University of Science and Technology（香港科学与技术大学）；City University of Hong Kong（香港城市大学）
University of Delaware（特拉华大学）；Bosch Research North America（博世北美研究院）；Technical University of Munich（慕尼黑工业大学）
We propose a keypoint-based object-level SLAM framework that can provide globally consistent 6DoF pose estimates for symmetric and asymmetric objects alike. To the best of our knowledge, our system is among the first to utilize the camera pose information from SLAM to provide prior knowledge for tracking keypoints on symmetric objects -- ensuring that new measurements are consistent with the current 3D scene. Moreover, our semantic keypoint network is trained to predict the Gaussian covariance for the keypoints that captures the true error of the prediction, and thus is not only useful as a we…
A Sensor-Aware Phenomenological Framework for Lidar Degradation Simulation and SLAM Robustness Evaluation
Turku Intelligent Embedded and Robotic Systems (TIERS) Lab , University of Turku（图尔库智能嵌入与机器人系统实验室，图尔库大学）；School of Information Science and Technology, Fudan University（信息科学与技术学院，复旦大学）
Graph-based SLAM-Aware Exploration with Prior Topo-Metric Information
School of Electrical and Electronic Engineering, Nanyang Technological University（南洋理工大学电气与电子工程学院）；Institute for Infocomm Research (I2R), Agency for Science, Technology and Research (A*STAR)（新加坡科技研究局信息通信研究所）
提出一种基于先验拓扑度量图的 SLAM 感知探索规划方法，通过全局回环增强路径并以贪心算法剪枝，在保持探索效率的同时提升建图精度。
In this paper, we present the RISE-SLAM algorithm for performing visual-inertial simultaneous localization and mapping (SLAM), while improving estimation consistency. Specifically, in order to achieve real-time operation, existing approaches often assume previously-estimated states to be perfectly known, which leads to inconsistent estimates. Instead, based on the idea of the Schmidt-Kalman filter, which has processing cost linear in the size of the state vector but quadratic memory requirements, we derive a new consistent approximate method in the information domain, which has linear memory r…
RICH-SLAM: Radar SLAM with Incremental and Continuous Hilbert Mapping
State Key Laboratory of Ocean Sensing, Zhejiang University（浙江大学海洋传感国家重点实验室）；Interdisciplinary Student Training Platform for Marine areas, Zhejiang University（浙江大学海洋交叉学科学生培养平台）；Department of Electronic and Computer Engineering, The Hong Kong University of Science and Technology（香港科技大学电子与计算机工程系）；School of AI and Robotics, Hunan University（湖南大学人工智能与机器人学院）；Ocean College, Zhejiang University（浙江大学海洋学院）；Institute of Deep-Sea Science and Engineering, Chinese Academy of Sciences（中国科学院深海科学与工程研究所）
TAIL: A Terrain-Aware Multi-Modal SLAM Dataset for Robot Locomotion in Deformable Granular Environments
Southern University of Science and Technology (SUSTech)（南方科技大学）；Shenzhen Key Laboratory of Biomimetic Robotics and Intelligent Systems（深圳市仿生机器人与智能系统重点实验室）；Guangdong Provincial Key Laboratory of Human-Augmentation and Rehabilitation Robotics in Universities（广东省高校人机增强与康复机器人重点实验室）
Autonomous Systems Lab, ETH Zürich（苏黎世联邦理工学院自主系统实验室）；Sevensense Robotics AG
Highly dynamic environments, with moving objects such as cars or humans, can pose a performance challenge for LiDAR SLAM systems that assume largely static scenes. To overcome this challenge and support the deployment of robots in real world scenarios, we propose a complete solution for a dynamic object aware LiDAR SLAM algorithm. This is achieved by leveraging a real-time capable neural network that can detect dynamic objects, thus allowing our system to deal with them explicitly. To efficiently generate the necessary training data which is key to our approach, we present a novel end-to-end o…
HI-SLAM2: Geometry-Aware Gaussian SLAM for Fast Monocular Scene Reconstruction
Institute for Photogrammetry and Geoinformatics, University of Stuttgart（摄影测量与地理信息学研究所，斯图加特大学）；Technical University of Munich（慕尼黑技术大学）；Karlsruhe University of Applied Sciences（卡尔斯鲁厄应用科学大学）；Munich Center for Machine Learning（慕尼黑机器学习中心）
The University of Adelaide（阿德莱德大学）；Australian Centre for Robotic Vision（澳大利亚机器人视觉中心）；NVIDIA（英伟达）；Queensland University of Technology（昆士兰科技大学）
Simultaneous Localization And Mapping (SLAM) is a fundamental problem in mobile robotics. While point-based SLAM methods provide accurate camera localization, the generated maps lack semantic information. On the other hand, state of the art object detection methods provide rich information about entities present in the scene from a single image. This work marries the two and proposes a method for representing generic objects as quadrics which allows object detections to be seamlessly integrated in a SLAM framework. For scene coverage, additional dominant planar structures are modeled as infini…
MAGS-SLAM: Monocular Multi-Agent Gaussian Splatting SLAM for Geometrically and Photometrically Consistent Reconstruction
ETH Zurich（苏黎世联邦理工学院）；Harbin Engineering University（哈尔滨工程大学）；University of Liverpool（利物浦大学）；University of Macau（澳门大学）；University of Ottawa（多伦多大学）；Wuhan University（武汉大学）；Imperial College London（伦敦帝国学院）
IIIT Hyderabad（国际信息技术研究所海得拉巴分校）；University of Tartu（塔尔图大学）
We present UrbanFly: an uncertainty-aware real-time planning framework for quadrotor navigation in urban high-rise environments. A core aspect of UrbanFly is its ability to robustly plan directly on the sparse point clouds generated by a Monocular Visual Inertial SLAM (VINS) backend. It achieves this by using the sparse point clouds to build an uncertainty-integrated cuboid representation of the environment through a data-driven monocular plane segmentation network. Our chosen world model provides faster distance queries than the more common voxel-grid representation, and UrbanFly leverages th…
Passage-Aware Structural Mapping for RGB-D Visual SLAM
Interdisciplinary Centre for Security, Reliability, and Trust (SnT), University of Luxembourg（卢森堡大学安全、可靠性与信任跨学科研究中心）；University of Luxembourg（卢森堡大学）；Faculty of Science, Technology, and Medicine at the University of Luxembourg（卢森堡大学科学、技术与医学学院）；Universidad Politécnica de Madrid（马德里理工大学）
Australian Center for Robotic Vision（澳大利亚机器人视觉中心）；University of Adelaide（阿德莱德大学）
Simultaneous Localization And Mapping (SLAM) is a fundamental problem in mobile robotics. While sparse point-based SLAM methods provide accurate camera localization, the generated maps lack semantic information. On the other hand, state of the art object detection methods provide rich information about entities present in the scene from a single image. This work incorporates a real-time deep-learned object detector to the monocular SLAM framework for representing generic objects as quadrics that permit detections to be seamlessly integrated while allowing the real-time performance. Finer recon…
MBA-SLAM: Motion Blur Aware Gaussian Splatting SLAM
College of Computer Science and Technology at Zhejiang University（浙江大学计算机科学与技术学院）；School of Engineering at Westlake University（西湖大学工程学院）；School of Engineering, Westlake University（西湖大学工程学院）
Amazon Alexa AI（亚马逊Alexa人工智能）；University of Southern California（南加州大学）；University of California, San Diego（加州大学圣迭戈分校）；University of California, Los Angeles（加州大学洛杉矶分校）
Recent years have witnessed an emerging paradigm shift toward embodied artificial intelligence, in which an agent must learn to solve challenging tasks by interacting with its environment. There are several challenges in solving embodied multimodal tasks, including long-horizon planning, vision-and-language grounding, and efficient exploration. We focus on a critical bottleneck, namely the performance of planning and navigation. To tackle this challenge, we propose a Neural SLAM approach that, for the first time, utilizes several modalities for exploration, predicts an affordance-aware semanti…
WaterSplat-SLAM: Photorealistic Monocular SLAM in Underwater Environment
Department of Electronic Engineering, Tsinghua University（清华大学电子工程系）；Key Laboratory of Marine Robotics, Shenyang（沈阳海洋机器人重点实验室）；University of Chinese Academy of Sciences（中国科学院大学）；Department of Electronic and Computer Engineering, The Hong Kong University of Science and Technology（香港科技大学电子及计算机工程学系）
PG-SLAM: Photo-realistic and Geometry-aware RGB-D SLAM in Dynamic Environments
The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；Technical University of Munich（慕尼黑工业大学）；Shanghai Jiao Tong University（上海交通大学）
NRGS-SLAM: Monocular Non-Rigid SLAM for Endoscopy via Deformation-Aware 3D Gaussian Splatting
Department of Mechanical and Automation Engineering and T Stone Robotics Institute, The Chinese University of Hong Kong（机械与自动化工程系和T Stone机器人研究所，香港中文大学）；School of Automation and Intelligent Sensing, the State Key Laboratory of Avionics Integration and Aviation System of-Systems Synthesis, and Shanghai Key Laboratory of Navigation and Location Based Services, Shanghai Jiao Tong University（自动化与智能感知学院、航空系统集成与航空系统综合国家重点实验室、上海导航与定位服务重点实验室，上海交通大学）
School of Integrated Circuits, Tsinghua University（清华大学集成电路学院）；Tsinghua University（清华大学）
Simultaneous localization and mapping (SLAM) is a critical technology that enables autonomous robots to be aware of their surrounding environment. With the development of deep learning, SLAM systems can achieve a higher level of perception of the environment, including the semantic and text levels. However, current works are limited in their ability to achieve a natural-language level of perception of the world. To address this limitation, we propose LP-SLAM, the first language-perceptive SLAM system that leverages large language models (LLMs). LP-SLAM has two major features: (a) it can detect…
Hong Kong University of Science and Technology（香港科技大学）
Moving objects can greatly jeopardize the performance of a visual simultaneous localization and mapping (vSLAM) system which relies on the static-world assumption. Motion removal have seen successful on solving this problem. Two main streams of solutions are based on either geometry constraints or deep semantic segmentation neural network. The former rely on static majority assumption, and the latter require labor-intensive pixel-wise annotations. In this paper we propose to adopt a novel weakly-supervised semantic segmentation method. The segmentation mask is obtained from a CNN pre-trained w…
NASA Jet Propulsion Laboratory（美国国家航空航天局喷气推进实验室）；California Institute of Technology（加州理工学院）；Santa Clara University（圣克拉拉大学）；Polytechnic University of Bari（巴里理工大学）
Enabling fully autonomous robots capable of navigating and exploring large-scale, unknown and complex environments has been at the core of robotics research for several decades. A key requirement in autonomous exploration is building accurate and consistent maps of the unknown environment that can be used for reliable navigation. Loop closure detection, the ability to assert that a robot has returned to a previously visited location, is crucial for consistent mapping as it reduces the drift caused by error accumulation in the estimated robot trajectory. Moreover, in multi-robot systems, loop c…
Vehicles with prolonged autonomous missions have to maintain environment awareness by simultaneous localization and mapping (SLAM). Closed loop correction is substituted by interpolation in rigid body transformation space in order to systematically reduce the accumulated error over different scales. The computation is divided to an edge computed lightweight SLAM and iterative corrections in the cloud environment. Tree locations in the forest environment are sent via a potentially limited communication bandwidths. Data from a real forest site is used in the verification of the proposed algorith…
Australian National University（澳大利亚国立大学）；University of Sydney（悉尼大学）
The static world assumption is standard in most simultaneous localisation and mapping (SLAM) algorithms. Increased deployment of autonomous systems to unstructured dynamic environments is driving a need to identify moving objects and estimate their velocity in real-time. Most existing SLAM based approaches rely on a database of 3D models of objects or impose significant motion constraints. In this paper, we propose a new feature-based, model-free, object-aware dynamic SLAM algorithm that exploits semantic segmentation to allow estimation of motion of rigid objects in a scene without the need t…
Carnegie Mellon University（卡内基梅隆大学）；Australian National University（澳大利亚国立大学）
Dynamic Object-aware SLAM (DOS) exploits object-level information to enable robust motion estimation in dynamic environments. Existing methods mainly focus on identifying and excluding dynamic objects from the optimization. In this paper, we show that feature-based visual SLAM systems can also benefit from the presence of dynamic articulated objects by taking advantage of two observations: (1) The 3D structure of each rigid part of articulated object remains consistent over time; (2) The points on the same rigid part follow the same motion. In particular, we present AirDOS, a dynamic object-aw…
Recent Semantic SLAM methods combine classical geometry-based estimation with deep learning-based object detection or semantic segmentation. In this paper we evaluate the quality of semantic maps generated by state-of-the-art class- and instance-aware dense semantic SLAM algorithms whose codes are publicly available and explore the impacts both semantic segmentation and pose estimation have on the quality of semantic maps. We obtain these results by providing algorithms with ground-truth pose and/or semantic segmentation data available from simulated environments. We establish that semantic se…
