School of Computer Science, The University of Adelaide（阿德莱德大学计算机科学学院）；School of Electrical Engineering and Computer Science, Queensland University of Technology（昆士兰科技大学电气与计算机科学学院）
Álvaro Parra, Tat-Jun Chin, Anders Eriksson, Ian Reid
Bundle adjustment plays a vital role in feature-based monocular SLAM. In many modern SLAM pipelines, bundle adjustment is performed to estimate the 6DOF camera trajectory and 3D map (3D point cloud) from the input feature tracks. However, two fundamental weaknesses plague SLAM systems based on bundle adjustment. First, the need to carefully initialise bundle adjustment means that all variables, in particular the map, must be estimated as accurately as possible and maintained over time, which makes the overall algorithm cumbersome. Second, since estimating the 3D structure (which requires sufficient baseline) is inherent in bundle adjustment, the SLAM algorithm will encounter difficulties during periods of slow motion or pure rotational motion.
We propose a different SLAM optimisation core: instead of bundle adjustment, we conduct rotation averaging to incrementally optimise only camera orientations. Given the orientations, we estimate the camera positions and 3D points via a quasi-convex formulation that can be solved efficiently and globally optimally. Our approach not only obviates the need to estimate and maintain the positions and 3D map at keyframe rate (which enables simpler SLAM systems), it is also more capable of handling slow motions or pure rotational motions.
National Institute of Advanced Industrial Science and Technology (AIST)（产业技术综合研究所）；Tokyo Institute of Technology（东京工业大学）
This study proposes a privacy-preserving Visual SLAM framework for estimating camera poses and performing bundle adjustment with mixed line and point clouds in real time. Previous studies have proposed localization methods to estimate a camera pose using a line-cloud map for a single image or a reconstructed point cloud. These methods offer a scene privacy protection against the inversion attacks by converting a point cloud to a line cloud, which reconstruct the scene images from the point cloud. However, they are not directly applicable to a video sequence because they do not address computat…
Kyushu University（九州大学）；Nara Institute of Science and Technology（奈良先端科学技术大学院大学）；Fukuoka University（福冈大学）；The University of Tokyo（东京大学）
Visual SLAM systems targeting static scenes have been developed with satisfactory accuracy and robustness. Dynamic 3D object tracking has then become a significant capability in visual SLAM with the requirement of understanding dynamic surroundings in various scenarios including autonomous driving, augmented and virtual reality. However, performing dynamic SLAM solely with monocular images remains a challenging problem due to the difficulty of associating dynamic features and estimating their positions. In this paper, we present MOTSLAM, a dynamic visual SLAM system with the monocular configur…
Center for Robotics, University of Bonn（博恩大学机器人中心）；MAVLab, TU Delft（代尔夫特理工大学MAVLab）；Lamarr Institute for Machine Learning and Artificial Intelligence（兰玛尔人工智能与机器学习研究所）
SCE-SLAM: Scale-Consistent Monocular SLAM via Scene Coordinate Embeddings
School of Computer Science and Engineering, State Key Laboratory of Complex & Critical Software Environment, Jiangxi Research Institute, Beihang University（计算机科学与工程学院、复杂与关键软件环境国家重点实验室、江西研究院、北航）；Macquarie University（麦考瑞大学）；Beijing Key Laboratory of Semiconductor Neural Network Intelligent Sensing and Computing Technology（北京半导体神经网络智能感知与计算技术重点实验室）
NSL-SLAM: High-Fidelity Neural Structured-Light Depth for Practical SLAM and Reconstruction
Wangxuan Institute of Computer Technology, Peking University（北京大学王选计算机研究所）；School of Computer Science and Technology, Beihang University（北京航空航天大学计算机科学与技术学院）；Yuanpei College, Peking University（北京大学元培学院）
IIIT-Delhi（德里信息技术学院）；NEC Laboratories America, Inc.（NEC美国实验室公司）；University of California, San Diego（加利福尼亚大学圣迭戈分校）
Classical monocular Simultaneous Localization And Mapping (SLAM) and the recently emerging convolutional neural networks (CNNs) for monocular depth prediction represent two largely disjoint approaches towards building a 3D map of the surrounding environment. In this paper, we demonstrate that the coupling of these two by leveraging the strengths of each mitigates the other's shortcomings. Specifically, we propose a joint narrow and wide baseline based self-improving framework, where on the one hand the CNN-predicted depth is leveraged to perform pseudo RGB-D feature-based SLAM, leading to bett…
We propose a new SLAM system that uses the semantic segmentation of objects and structures in the scene. Semantic information is relevant as it contains high level information which may make SLAM more accurate and robust. Our contribution is twofold: i) A new SLAM system based on ORB-SLAM2 that creates a semantic map made of clusters of points corresponding to objects instances and structures in the scene. ii) A modification of the classical Bundle Adjustment formulation to constrain each cluster using geometrical priors, which improves both camera localization and reconstruction and enables a…
The joint optimization of the sensor trajectory and 3D map is a crucial characteristic of Simultaneous Localization and Mapping (SLAM) systems. To achieve this, the gold standard is Bundle Adjustment (BA). Modern 3D LiDARs now retain higher resolutions that enable the creation of point cloud images resembling those taken by conventional cameras. Nevertheless, the typical effective global refinement techniques employed for RGB-D sensors are not widely applied to LiDARs. This paper presents a novel BA photometric strategy that accounts for both RGB-D and LiDAR in the same way. Our work can be us…
3D reconstruction has been developing all these two decades, from moderate to medium size and to large scale. It's well known that bundle adjustment plays an important role in 3D reconstruction, mainly in Structure from Motion(SfM) and Simultaneously Localization and Mapping(SLAM). While bundle adjustment optimizes camera parameters and 3D points as a non-negligible final step, it suffers from memory and efficiency requirements in very large scale reconstruction. In this paper, we study the development of bundle adjustment elaborately in both conventional and distributed approaches. The detail…
DenseSplat: Densifying Gaussian Splatting SLAM with Neural Radiance Prior
Department of Computer Science, Dalian University of Technology（大连理工大学计算机科学系）；Department of Mechano-informatics, Information Science and Technology, The University of Tokyo（东京大学机械信息学系、信息科学与技术系）；Institute of Medical Robotics and Department of Automation, Shanghai Jiao Tong University（上海交通大学医学机器人研究所、自动化系）；Key Laboratory of System Control and Information Processing, Ministry of Education（教育部系统控制与信息处理重点实验室）
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；TianJin Intelligent Tech.Institute of CASIA Co. ,Ltd（中科智能（天津）科技研究院有限公司）
Accurate camera pose estimation result is essential for visual SLAM (VSLAM). This paper presents a novel pose correction method to improve the accuracy of the VSLAM system. Firstly, the relationship between the camera pose estimation error and bias values of map points is derived based on the optimized function in VSLAM. Secondly, the bias value of the map point is calculated by a statistical method. Finally, the camera pose estimation error is compensated according to the first derived relationship. After the pose correction, procedures of the original system, such as the bundle adjustment (B…
AASS research center, Örebro University, Sweden（奥雷布罗大学AASS研究中心）；Czech Technical University in Prague（布拉格捷克技术大学）；Technical University of Munich, Chair: Perception for Intelligent Systems（慕尼黑技术大学感知智能系统教研室）
HI-SLAM: Monocular Real-time Dense Mapping with Hybrid Implicit Fields
University of Stuttgart（斯图加特大学）；Huawei Munich Research Center（华为慕尼黑研究中心）；Central Media Technology Institute, Huawei 2012 Laboratories（华为2012实验室中央媒体技术研究院）；Technical University of Munich（慕尼黑工业大学）
This paper presents a hybrid real-time camera pose estimation framework with a novel partitioning scheme and introduces motion averaging to monocular Simultaneous Localization and Mapping (SLAM) systems. Breaking through the limitations of fixed-size temporal partitioning in many conventional SLAM pipelines, our approach significantly improves the accuracy of local bundle adjustment by gathering spatially-strongly-connected cameras into each block. With the dynamic initialization using intermediate computation values, \XL{we improve the Levenberg-Marquardt solver to further enhance the efficie…
SemGauss-SLAM: Dense Semantic Gaussian Splatting SLAM
School of Automation and Intelligent Sensing, Key Laboratory of System Control and Information Processing, Ministry of Education of China, Shanghai Jiao Tong University, Shanghai（自动化与智能感知学院、系统控制与信息处理重点实验室、中华人民共和国教育部、上海交通大学、上海）；University of Cambridge, United Kingdom（剑桥大学、英国）
HI-SLAM2: Geometry-Aware Gaussian SLAM for Fast Monocular Scene Reconstruction
Institute for Photogrammetry and Geoinformatics, University of Stuttgart（摄影测量与地理信息学研究所，斯图加特大学）；Technical University of Munich（慕尼黑技术大学）；Karlsruhe University of Applied Sciences（卡尔斯鲁厄应用科学大学）；Munich Center for Machine Learning（慕尼黑机器学习中心）
MCGS-SLAM: A Multi-Camera SLAM Framework Using Gaussian Splatting for High-Fidelity Mapping
Department of Mathematics, ETH Zurich（苏黎世联邦理工学院数学系）；Department of Mechanical and Process Engineering, ETH Zurich（苏黎世联邦理工学院机械与工艺工程系）；Department of Informatics, University of Zurich（苏黎世大学信息系）；Institute for Photogrammetry, University of Stuttgart（斯图加特大学摄影测量研究所）；Computer Vision and Geometry Group, ETH Zurich（苏黎世联邦理工学院计算机视觉与几何组）；Microsoft Spatial AI Lab（微软空间AI实验室）；Computer Vision Research Group, University of Amsterdam（阿姆斯特丹大学计算机视觉研究组）
Robotics Institute, Carnegie Mellon University（卡内基梅隆大学机器人研究所）
We present a method for single image 3D cuboid object detection and multi-view object SLAM in both static and dynamic environments, and demonstrate that the two parts can improve each other. Firstly for single image object detection, we generate high-quality cuboid proposals from 2D bounding boxes and vanishing points sampling. The proposals are further scored and selected based on the alignment with image edges. Secondly, multi-view bundle adjustment with new object measurements is proposed to jointly optimize poses of cameras, objects and points. Objects can provide long-range geometric and …
RoSe-SLAM: Robust Semantic-Aware Gaussian Splatting SLAM from Dynamic Monocular Videos
The Chinese University of Hong Kong（香港中文大学）；The University of Manchester（曼彻斯特大学）；Centre for Perceptual and Interactive Intelligence (CPII) Limited（感知与互动智能中心有限公司）
Compact 3D Gaussian Splatting For Dense Visual SLAM
Shanghai Jiao Tong University（上海交通大学）；Nanyang Technological University（南洋理工大学）；The University of Tokyo（东京大学）；Harvard University（哈佛大学）；University of Cambridge（剑桥大学）
Hong Kong University of Science and Technology（香港科技大学）；Everest Innovation Technology（珠峰创新科技）
Current bundle adjustment solvers such as the Levenberg-Marquardt (LM) algorithm are limited by the bottleneck in solving the Reduced Camera System (RCS) whose dimension is proportional to the camera number. When the problem is scaled up, this step is neither efficient in computation nor manageable for a single compute node. In this work, we propose a stochastic bundle adjustment algorithm which seeks to decompose the RCS approximately inside the LM iterations to improve the efficiency and scalability. It first reformulates the quadratic programming problem of an LM iteration based on the clus…
We introduce DROID-SLAM, a new deep learning based SLAM system. DROID-SLAM consists of recurrent iterative updates of camera pose and pixelwise depth through a Dense Bundle Adjustment layer. DROID-SLAM is accurate, achieving large improvements over prior work, and robust, suffering from substantially fewer catastrophic failures. Despite training on monocular video, it can leverage stereo or RGB-D video to achieve improved performance at test time. The URL to our open source code is https://github.com/princeton-vl/DROID-SLAM.
Visual SLAM shows significant progress in recent years due to high attention from vision community but still, challenges remain for low-textured environments. Feature based visual SLAMs do not produce reliable camera and structure estimates due to insufficient features in a low-textured environment. Moreover, existing visual SLAMs produce partial reconstruction when the number of 3D-2D correspondences is insufficient for incremental camera estimation using bundle adjustment. This paper presents Edge SLAM, a feature based monocular visual SLAM which mitigates the above mentioned problems. Our p…
LiDAR odometry is one of the essential parts of LiDAR simultaneous localization and mapping (SLAM). However, existing LiDAR odometry tends to match a new scan simply iteratively with previous fixed-pose scans, gradually accumulating errors. Furthermore, as an effective joint optimization mechanism, bundle adjustment (BA) cannot be directly introduced into real-time odometry due to the intensive computation of large-scale global landmarks. Therefore, this letter designs a new strategy named a landmark map for bundle adjustment odometry (LMBAO) in LiDAR SLAM to solve these problems. First, BA-ba…
Advancing Dense Endoscopic Reconstruction with Gaussian Splatting-driven Surface Normal-aware Tracking and Mapping
The Chinese University of Hong Kong (CUHK)（香港中文大学）；CUHK Shenzhen Research Institute（香港中文大学深圳研究院）；Hong Kong Institute of Science & Innovation, Chinese Academy of Sciences（中国科学院香港创新研究院）；Qilu Hospital of Shandong University（山东大学齐鲁医院）
Physically-Based Photometric Bundle Adjustment in Non-Lambertian Environments
Technical University of Munich（慕尼黑工业大学）；Munich Center for Machine Learning（慕尼黑机器学习中心）；The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；The Chinese University of Hong Kong（香港中文大学）
RootSLAM: Square Root Bundle Adjustment for Visual-Inertial SLAM
Technical University of Munich（慕尼黑工业大学）；Munich Center for Machine Learning（慕尼黑机器学习中心）；National University of Rosario（罗萨里奥国立大学）；CIFASIS (CONICET-UNR)（CIFASIS（CONICET-UNR））
We propose a novel algorithm for the joint refinement of structure and motion parameters from image data directly without relying on fixed and known correspondences. In contrast to traditional bundle adjustment (BA) where the optimal parameters are determined by minimizing the reprojection error using tracked features, the proposed algorithm relies on maximizing the photometric consistency and estimates the correspondences implicitly. Since the proposed algorithm does not require correspondences, its application is not limited to corner-like structure; any pixel with nonvanishing gradient coul…
Carnegie Mellon University（卡内基梅隆大学）；Massachusetts Institute of Technology（麻省理工学院）；The University of Adelaide（阿德莱德大学）
Neural Radiance Fields (NeRF) have recently gained a surge of interest within the computer vision community for its power to synthesize photorealistic novel views of real-world scenes. One limitation of NeRF, however, is its requirement of accurate camera poses to learn the scene representations. In this paper, we propose Bundle-Adjusting Neural Radiance Fields (BARF) for training NeRF from imperfect (or even unknown) camera poses -- the joint problem of learning neural 3D representations and registering camera frames. We establish a theoretical connection to classical image alignment and show…
Robot Innovation Research Center（机器人创新研究中心）；National Institute of Advanced Industrial Science and Technology (AIST)（日本产业技术综合研究所（AIST））
In this paper, we propose a novel indirect monocular SLAM algorithm called "VITAMIN-E," which is highly accurate and robust as a result of tracking extremely dense feature points. Typical indirect methods have difficulty in reconstructing dense geometry because of their careful feature point selection for accurate matching. Unlike conventional methods, the proposed method processes an enormous number of feature points by tracking the local extrema of curvature informed by dominant flow estimation. Because this may lead to high computational cost during bundle adjustment, we propose a novel opt…
This paper presents an efficient algorithm for the least-squares problem using the point-to-plane cost, which aims to jointly optimize depth sensor poses and plane parameters for 3D reconstruction. We call this least-squares problem \textbf{Planar Bundle Adjustment} (PBA), due to the similarity between this problem and the original Bundle Adjustment (BA) in visual reconstruction. As planes ubiquitously exist in the man-made environment, they are generally used as landmarks in SLAM algorithms for various depth sensors. PBA is important to reduce drift and improve the quality of the map. However…
University of Malaga（马拉加大学）；University of Zurich（苏黎世大学）；ETH Zurich（苏黎世联邦理工学院）
Traditional approaches to stereo visual SLAM rely on point features to estimate the camera trajectory and build a map of the environment. In low-textured environments, though, it is often difficult to find a sufficient number of reliable point features and, as a consequence, the performance of such algorithms degrades. This paper proposes PL-SLAM, a stereo visual SLAM system that combines both points and line segments to work robustly in a wider variety of scenarios, particularly in those where point features are scarce or not well-distributed in the image. PL-SLAM leverages both points and se…
Samsung R&D Institute China-Beijing（三星中国研发中心北京研究所）；The Hong Kong University of Science and Technology（香港科技大学）；Samsung Advanced Institute of Technology（三星先进技术研究院）
Facebook Reality Lab（Facebook现实实验室）；School of Electrical and Computer Engineering, Georgia Institute of Technology（佐治亚理工学院电气与计算机工程学院）；Institute of Robotics and Intelligent Machines, Georgia Institute of Technology（佐治亚理工学院机器人与智能机器研究所）
The cost-efficiency of visual(-inertial) SLAM (VSLAM) is a critical characteristic of resource-limited applications. While hardware and algorithm advances have been significantly improved the cost-efficiency of VSLAM front-ends, the cost-efficiency of VSLAM back-ends remains a bottleneck. This paper describes a novel, rigorous method to improve the cost-efficiency of local BA in a BA-based VSLAM back-end. An efficient algorithm, called Good Graph, is developed to select size-reduced graphs optimized in local BA with condition preservation. To better suit BA-based VSLAM back-ends, the Good Grap…
