Object SLAM is considered increasingly significant for robot high-level perception and decision-making. Existing studies fall short in terms of data association, object representation, and semantic mapping and frequently rely on additional assumptions, limiting their performance. In this paper, we present a comprehensive object SLAM framework that focuses on object-based perception and object-oriented robot tasks. First, we propose an ensemble data association approach for associating objects in complicated conditions by incorporating parametric and nonparametric statistic testing. In addition, we suggest an outlier-robust centroid and scale estimation algorithm for modeling objects based on the iForest and line alignment. Then a lightweight and object-oriented map is represented by estimated general object models. Taking into consideration the semantic invariance of objects, we convert the object map to a topological map to provide semantic descriptors to enable multi-map matching. Finally, we suggest an object-driven active exploration strategy to achieve autonomous mapping in the grasping scenario. A range of public datasets and real-world results in mapping, augmented reality, scene matching, relocalization, and robotic manipulation have been used to evaluate the proposed object SLAM framework for its efficient performance.
Simultaneous mapping and localization (SLAM) in an real indoor environment is still a challenging task. Traditional SLAM approaches rely heavily on low-level geometric constraints like corners or lines, which may lead to tracking failure in textureless surroundings or cluttered world with dynamic objects. In this paper, a compact semantic SLAM framework is proposed, with utilization of both geometric and object-level semantic constraints jointly, a more consistent mapping result, and more accurate pose estimation can be obtained. Two main contributions are presented int the paper, a) a robust …
It is often desirable to capture and map semantic information of an environment during simultaneous localization and mapping (SLAM). Such semantic information can enable a robot to better distinguish places with similar low-level geometric and visual features and perform high-level tasks that use semantic information about objects to be manipulated and environments to be navigated. While semantic SLAM has gained increasing attention, there is little research on semanticlevel data association based on semantic objects, i.e., object-level data association. In this paper, we propose a novel objec…
Mapping and self-localization in unknown environments are fundamental capabilities in many robotic applications. These tasks typically involve the identification of objects as unique features or landmarks, which requires the objects both to be detected and then assigned a unique identifier that can be maintained when viewed from different perspectives and in different images. The \textit{data association} and \textit{simultaneous localization and mapping} (SLAM) problems are, individually, well-studied in the literature. But these two problems are inherently tightly coupled, and that has not b…
Faculty of Robot Science and Engineering, Northeastern University（东北大学机器人科学与工程学院）；College of Information Science and Engineering, Northeastern University（东北大学信息科学与工程学院）；The Chinese University of Hong Kong（香港中文大学）
Object-level data association and pose estimation play a fundamental role in semantic SLAM, which remain unsolved due to the lack of robust and accurate algorithms. In this work, we propose an ensemble data associate strategy for integrating the parametric and nonparametric statistic tests. By exploiting the nature of different statistics, our method can effectively aggregate the information of different measurements, and thus significantly improve the robustness and accuracy of data association. We then present an accurate object pose estimation framework, in which an outliers-robust centroid…
PAS-SLAM: A Visual SLAM System for Planar Ambiguous Scenes
School of Electronic Information and Electrical Engineering, Dalian University of Technology（大连理工大学电子信息与电气工程学部）；Department of Automation, Tsinghua University（清华大学自动化系）；School of Electronic and Computer Engineering, Peking University（北京大学电子与计算机工程学院）；College of Information Science and Engineering, Northeastern University（东北大学信息科学与工程学院）
Max Planck Institute for Intelligent Systems（马克斯·普朗克智能系统研究所）
The majority of approaches for acquiring dense 3D environment maps with RGB-D cameras assumes static environments or rejects moving objects as outliers. The representation and tracking of moving objects, however, has significant potential for applications in robotics or augmented reality. In this paper, we propose a novel approach to dynamic SLAM with dense object-level representations. We represent rigid objects in local volumetric signed distance function (SDF) maps, and formulate multi-object tracking as direct alignment of RGB-D images with the SDF representations. Our main novelty is a pr…
University of Bristol（布里斯托大学）；Zhejiang University（浙江大学）
We propose a novel end-to-end RGB-D SLAM, iDF-SLAM, which adopts a feature-based deep neural tracker as the front-end and a NeRF-style neural implicit mapper as the back-end. The neural implicit mapper is trained on-the-fly, while though the neural tracker is pretrained on the ScanNet dataset, it is also finetuned along with the training of the neural implicit mapper. Under such a design, our iDF-SLAM is capable of learning to use scene-specific features for camera tracking, thus enabling lifelong learning of the SLAM system. Both the training for the tracker and the mapper are self-supervised…
National Taiwan University（国立台湾大学）；National Yang Ming Chiao Tung University（国立阳明交通大学）；Mobile Drive Technology（摩智科技）
A spatial AI that can perform complex tasks through visual signals and cooperate with humans is highly anticipated. To achieve this, we need a visual SLAM that easily adapts to new scenes without pre-training and generates dense maps for downstream tasks in real-time. None of the previous learning-based and non-learning-based visual SLAMs satisfy all needs due to the intrinsic limitations of their components. In this work, we develop a visual SLAM named Orbeez-SLAM, which successfully collaborates with implicit neural representation and visual odometry to achieve our goals. Moreover, Orbeez-SL…
MoD-SLAM: Monocular Dense Mapping for Unbounded 3D Scene Reconstruction
Columbia University（哥伦比亚大学）；Cloudspace Technology（云空间科技）；The University of Tokyo（东京大学）；Dianjing Ciyuan Culture（电境慈源文化）；Dalian University of Technology（大连理工大学）
Dyson Technology Ltd.（戴森技术有限公司）；Imperial College London（帝国理工学院）
We propose a novel dense mapping framework for sparse visual SLAM systems which leverages a compact scene representation. State-of-the-art sparse visual SLAM systems provide accurate and reliable estimates of the camera trajectory and locations of landmarks. While these sparse maps are useful for localization, they cannot be used for other tasks such as obstacle avoidance or scene understanding. In this paper we propose a dense mapping framework to complement sparse visual SLAM systems which takes as input the camera poses, keyframes and sparse points produced by the SLAM system and predicts a…
Existing solutions to visual simultaneous localization and mapping (V-SLAM) assume that errors in feature extraction and matching are independent and identically distributed (i.i.d), but this assumption is known to not be true -- features extracted from low-contrast regions of images exhibit wider error distributions than features from sharp corners. Furthermore, V-SLAM algorithms are prone to catastrophic tracking failures when sensed images include challenging conditions such as specular reflections, lens flare, or shadows of dynamic objects. To address such failures, previous work has focus…
Structure from Motion (SfM) techniques are being increasingly used to create 3D maps from images in many domains including environmental monitoring. However, SfM techniques are often confounded in visually repetitive environments as they rely primarily on globally distinct image features. Simultaneous Localization and Mapping (SLAM) techniques offer a potential solution in visually repetitive environments since they use local feature matching, but SLAM approaches work best with wide-angle cameras that are often unsuitable for documenting the environmental system of interest. We resolve this is…
Nanjing University of Science and Technology（南京理工大学）
Simultaneous Localization and Mapping (SLAM) plays an important role in outdoor and indoor applications ranging from autonomous driving to indoor robotics. Outdoor SLAM has been widely used with the assistance of LiDAR or GPS. For indoor applications, the LiDAR technique does not satisfy the accuracy requirement and the GPS signals will be lost. An accurate and efficient scene sensing technique is required for indoor SLAM. As the most promising 3D sensing technique, the opportunities for indoor SLAM with fringe projection profilometry (FPP) systems are obvious, but methods to date have not ful…
MCGS-SLAM: A Multi-Camera SLAM Framework Using Gaussian Splatting for High-Fidelity Mapping
Department of Mathematics, ETH Zurich（苏黎世联邦理工学院数学系）；Department of Mechanical and Process Engineering, ETH Zurich（苏黎世联邦理工学院机械与工艺工程系）；Department of Informatics, University of Zurich（苏黎世大学信息系）；Institute for Photogrammetry, University of Stuttgart（斯图加特大学摄影测量研究所）；Computer Vision and Geometry Group, ETH Zurich（苏黎世联邦理工学院计算机视觉与几何组）；Microsoft Spatial AI Lab（微软空间AI实验室）；Computer Vision Research Group, University of Amsterdam（阿姆斯特丹大学计算机视觉研究组）
SemGauss-SLAM: Dense Semantic Gaussian Splatting SLAM
School of Automation and Intelligent Sensing, Key Laboratory of System Control and Information Processing, Ministry of Education of China, Shanghai Jiao Tong University, Shanghai（自动化与智能感知学院、系统控制与信息处理重点实验室、中华人民共和国教育部、上海交通大学、上海）；University of Cambridge, United Kingdom（剑桥大学、英国）
DFKI - German Research Center for Artificial Intelligence（德国人工智能研究中心）
This paper demonstrates a system capable of combining a sparse, indirect, monocular visual SLAM, with both offline and real-time Multi-View Stereo (MVS) reconstruction algorithms. This combination overcomes many obstacles encountered by autonomous vehicles or robots employed in agricultural environments, such as overly repetitive patterns, need for very detailed reconstructions, and abrupt movements caused by uneven roads. Furthermore, the use of a monocular SLAM makes our system much easier to integrate with an existing device, as we do not rely on a LiDAR (which is expensive and power consum…
DFKI - German Research Center for Artificial Intelligence（德国人工智能研究中心）
This paper presents a visual SLAM system that uses both points and lines for robust camera localization, and simultaneously performs a piece-wise planar reconstruction (PPR) of the environment to provide a structural map in real-time. One of the biggest challenges in parallel tracking and mapping with a monocular camera is to keep the scale consistent when reconstructing the geometric primitives. This further introduces difficulties in graph optimization of the bundle adjustment (BA) step. We solve these problems by proposing several run-time optimizations on the reconstructed lines and planes…
AgriGS-SLAM: Orchard Mapping Across Seasons via Multi-View Gaussian Splatting SLAM
Dipartimento di Bioingegneria, Elettronica e Informazione, Politecnico di Milano（生物工程、电子与信息系，米兰理工大学）；Agricultural Biosystems Engineering, Wageningen University & Research（农业生物系统工程，瓦赫宁根大学与研究）
Northeastern University（东北大学）；The Chinese University of Hong Kong（香港中文大学）；Ulster University（阿尔斯特大学）
This paper presents the first active object mapping framework for complex robotic manipulation and autonomous perception tasks. The framework is built on an object SLAM system integrated with a simultaneous multi-object pose estimation process that is optimized for robotic grasping. Aiming to reduce the observation uncertainty on target objects and increase their pose estimation accuracy, we also design an object-driven exploration strategy to guide the object mapping process, enabling autonomous mapping and high-level perception. Combining the mapping module and the exploration strategy, an a…
Neural field-based 3D representations have recently been adopted in many areas including SLAM systems. Current neural SLAM or online mapping systems lead to impressive results in the presence of simple captures, but they rely on a world-centric map representation as only a single neural field model is used. To define such a world-centric representation, accurate and static prior information about the scene, such as its boundaries and initial camera poses, are required. However, in real-time and on-the-fly scene capture applications, this prior knowledge cannot be assumed as fixed or static, si…
Southern University of Science and Technology（南方科技大学）
Structured Light Illumination (SLI) systems have been used for reliable indoor dense 3D scanning via phase triangulation. However, mobile SLI systems for 360 degree 3D reconstruction demand 3D point cloud registration, involving high computational complexity. In this paper, we propose a phase based Simultaneous Localization and Mapping (Phase-SLAM) framework for fast and accurate SLI sensor pose estimation and 3D object reconstruction. The novelty of this work is threefold: (1) developing a reprojection model from 3D points to 2D phase data towards phase registration with low computational com…
Object-oriented SLAM is a popular technology in autonomous driving and robotics. In this paper, we propose a stereo visual SLAM with a robust quadric landmark representation method. The system consists of four components, including deep learning detection, object-oriented data association, dual quadric landmark initialization and object-based pose optimization. State-of-the-art quadric-based SLAM algorithms always face observation related problems and are sensitive to observation noise, which limits their application in outdoor scenes. To solve this problem, we propose a quadric initialization…
BDIS-SLAM: A lightweight CPU-based dense stereo SLAM for surgery
United Imaging Research Institute of Intelligent Imaging（联影智能影像研究院）；University of Michigan（密歇根大学）；University of Technology Sydney（悉尼科技大学）；Imperial College London（伦敦帝国理工学院）
GLAM-SLAM: Real-time Gaussian Large-scale Mapping via Flow Densification and Spatial Decomposition
HERON - Hellenic Robotics Center of Excellence, Athena Research Center（希腊机器人卓越中心 - 阿西娜研究中心）；Institute of Robotics, Athena Research Center（阿西娜研究中心机器人研究所）；National Technical University of Athens（雅典国立技术大学）
BPDA-GMM: Bayesian Probabilistic Data Association via Gaussian Mixture Models for Semantic SLAM
School of Information Science, Japan Advanced Institute of Science and Technology（信息科学学系，日本科学技术大学）；University of Engineering and Technology, Vietnam National University（工程技术大学，越南国家大学）
Using pole and plane objects in lidar SLAM can increase accuracy and decrease map storage requirements compared to commonly-used point cloud maps. However, place recognition and geometric verification using these landmarks is challenging due to the requirement for global matching without an initial guess. Existing works typically only leverage either pole or plane landmarks, limiting application to a restricted set of environments. We present a global data association method for loop closure in lidar scans using 3D line and plane objects simultaneously and in a unified manner. The main novelty…
Massachusetts Institute of Technology (MIT)（麻省理工学院）；University of Massachusetts Amherst (UMass Amherst)（马萨诸塞大学阿默斯特分校）
Modern robotic systems sense the environment geometrically, through sensors like cameras, lidar, and sonar, as well as semantically, often through visual models learned from data, such as object detectors. We aim to develop robots that can use all of these sources of information for reliable navigation, but each is corrupted by noise. Rather than assume that object detection will eventually achieve near perfect performance across the lifetime of a robot, in this work we represent and cope with the semantic and geometric uncertainty inherent in methods like object detection. Specifically, we mo…
Korea Advanced Institute of Science and Technology (KAIST)（韩国科学技术院）
In feature-based simultaneous localization and mapping (SLAM), line features complement the sparsity of point features, making it possible to map the surrounding environment structure. Existing approaches utilizing line features have primarily employed a measurement model that uses line re-projection. However, the direction vectors used in the 3D line mapping process cannot be corrected because the line measurement model employs only the lines' normal vectors in the Plücker coordinate. As a result, problems like degeneracy that occur during the 3D line mapping process cannot be solved. To tack…
MMD-SLAM: Structure-Enhanced Multi-Meta Gaussian Distribution-Guided Visual SLAM
HFIPS, Chinese Academy of Sciences（中国科学院合肥物质科学研究院）；University of Science and Technology of China（中国科学技术大学）；Aarhus University（奥胡斯大学）；University of Tokyo（东京大学）；Beijing University of Chemical Technology（北京化工大学）；North China Electric Power University（华北电力大学）
HI-SLAM: Monocular Real-time Dense Mapping with Hybrid Implicit Fields
University of Stuttgart（斯图加特大学）；Huawei Munich Research Center（华为慕尼黑研究中心）；Central Media Technology Institute, Huawei 2012 Laboratories（华为2012实验室中央媒体技术研究院）；Technical University of Munich（慕尼黑工业大学）
Imperial College London（伦敦帝国理工学院）；Dyson Technology Ltd.（戴森科技有限公司）
There are many possibilities for how to represent the map in simultaneous localisation and mapping (SLAM). While sparse, keypoint-based SLAM systems have achieved impressive levels of accuracy and robustness, their maps may not be suitable for many robotic tasks. Dense SLAM systems are capable of producing dense reconstructions, but can be computationally expensive and, like sparse systems, lack higher-level information about the structure of a scene. Human-made environments contain a lot of structure, and we seek to take advantage of this by enabling the use of quadric surfaces as features in…
Hong Kong University of Science and Technology（香港科技大学）；University of California, San Diego（加利福尼亚大学圣地亚哥分校）；ATG Robotics（ATG机器人技术公司）；University of Bristol（布里斯托大学）；Bristol Robotics Laboratory（布里斯托机器人实验室）
In this paper, we present a multi-agent framework for real-time large-scale 3D reconstruction applications. In SLAM, researchers usually build and update a 3D map after applying non-linear pose graph optimization techniques. Moreover, many multi-agent systems are prevalently using odometry information from additional sensors. These methods generally involve intensive computer vision algorithms and are tightly coupled with various sensors. We develop a generic method for the keychallenging scenarios in multi-agent 3D mapping based on different camera systems. The proposed framework performs act…
University of Melbourne（墨尔本大学）；University of Texas at Dallas（德克萨斯大学达拉斯分校）；Ecole Polytechnique Federale de Lausanne（洛桑联邦理工学院）；Australian National University（澳大利亚国立大学）
With advances in image processing and machine learning, it is now feasible to incorporate semantic information into the problem of simultaneous localisation and mapping (SLAM). Previously, SLAM was carried out using lower level geometric features (points, lines, and planes) which are often view-point dependent and error prone in visually repetitive environments. Semantic information can improve the ability to recognise previously visited locations, as well as maintain sparser maps for long term SLAM applications. However, SLAM in repetitive environments has the critical problem of assigning me…
Existing simultaneous localization and mapping (SLAM) algorithms are not robust in challenging low-texture environments because there are only few salient features. The resulting sparse or semi-dense map also conveys little information for motion planning. Though some work utilize plane or scene layout for dense map regularization, they require decent state estimation from other sources. In this paper, we propose real-time monocular plane SLAM to demonstrate that scene understanding could improve both state estimation and dense mapping especially in low-texture environments. The plane measurem…
Multicam-SLAM: Non-overlapping Multi-camera SLAM for Indirect Visual Localization and Navigation
School of Electronic Information and Electrical Engineering, Shanghai Jiao Tong University（上海交通大学电子信息与电气工程学院）；School of Mechanical and Power Engineering, East China University of Science and Technology（华东理工大学机械与动力工程学院）
Massachusetts Institute of Technology（麻省理工学院）；Laboratory for Information & Decision Systems（信息与决策系统实验室）
We provide an open-source C++ library for real-time metric-semantic visual-inertial Simultaneous Localization And Mapping (SLAM). The library goes beyond existing visual and visual-inertial SLAM libraries (e.g., ORB-SLAM, VINS- Mono, OKVIS, ROVIO) by enabling mesh reconstruction and semantic labeling in 3D. Kimera is designed with modularity in mind and has four key components: a visual-inertial odometry (VIO) module for fast and accurate state estimation, a robust pose graph optimizer for global trajectory estimation, a lightweight 3D mesher module for fast mesh reconstruction, and a dense 3D…
