In this paper, we present the RISE-SLAM algorithm for performing visual-inertial simultaneous localization and mapping (SLAM), while improving estimation consistency. Specifically, in order to achieve real-time operation, existing approaches often assume previously-estimated states to be perfectly known, which leads to inconsistent estimates. Instead, based on the idea of the Schmidt-Kalman filter, which has processing cost linear in the size of the state vector but quadratic memory requirements, we derive a new consistent approximate method in the information domain, which has linear memory requirements and adjustable (constant to linear) processing cost. In particular, this method, the resource-aware inverse Schmidt estimator (RISE), allows trading estimation accuracy for computational efficiency. Furthermore, and in order to better address the requirements of a SLAM system during an exploration vs. a relocalization phase, we employ different configurations of RISE (in terms of the number and order of states updated) to maximize accuracy while preserving efficiency. Lastly, we evaluate the proposed RISE-SLAM algorithm on publicly-available datasets and demonstrate its superiority, both in terms of accuracy and efficiency, as compared to alternative visual-inertial SLAM systems.
It holds great implications for practical applications to enable centimeter-accuracy positioning for mobile and wearable sensor systems. In this paper, we propose a novel, high-precision, efficient visual-inertial (VI)-SLAM algorithm, termed Schmidt-EKF VI-SLAM (SEVIS), which optimally fuses IMU measurements and monocular images in a tightly-coupled manner to provide 3D motion tracking with bounded error. In particular, we adapt the Schmidt Kalman filter formulation to selectively include informative features in the state vector while treating them as nuisance parameters (or Schmidt states) on…
University of Technology Sydney（悉尼科技大学）；Queensland University of Technology（昆士兰科技大学）
Visual-inertial SLAM has been studied widely due to the advantage of its lightweight, cost-effectiveness, and rich information compared to other sensors. A multi-state constrained filter (MSCKF) and its Schmidt version have been developed to address the computational cost, which treats keyframes as static nuisance parameters, leading to sub-optimal performance. We propose a new Compressed-MSCKF which can achieve improved accuracy with moderate computational costs. By keeping the information gain with compressed form, it can limit to $\mathcal{O}(L)$ with $L$ being the number of local keyframes…
MoCap2GT: A High-Precision Ground Truth Estimator for SLAM Benchmarking Based on Motion Capture and IMU Fusion
Shanghai Institute of Technical Physics, Chinese Academy of Sciences（上海技术物理研究所，中国科学院）；Advanced Display and Sensing Research Center, Yongjiang Laboratory（先进显示与传感研究中心，永江实验室）
Simultaneous Localization and Mapping (SLAM) is one of the key robotics tasks as it tackles simultaneous mapping of the unknown environment defined by multiple landmark positions and localization of the unknown pose (i.e., attitude and position) of the robot in three-dimensional (3D) space. The true SLAM problem is modeled on the Lie group of $\mathbb{SLAM}_{n}\left(3\right)$, and its true dynamics rely on angular and translational velocities. This paper proposes a novel geometric nonlinear stochastic estimator algorithm for SLAM on $\mathbb{SLAM}_{n}\left(3\right)$ that precisely mimics the n…
Computer Science and Artificial Intelligence Laboratory, Massachusetts Institute of Technology（麻省理工学院计算机科学与人工智能实验室）；Laboratory for Information and Decision Systems, Massachusetts Institute of Technology（麻省理工学院信息与决策系统实验室）
In this work we present the first initialization methods equipped with explicit performance guarantees adapted to the pose-graph simultaneous localization and mapping (SLAM) and rotation averaging (RA) problems. SLAM and rotation averaging are typically formalized as large-scale nonconvex point estimation problems, with many bad local minima that can entrap the smooth optimization methods typically applied to solve them; the performance of standard SLAM and RA algorithms thus crucially depends upon the quality of the estimates used to initialize this local search. While many initialization met…
In this paper, we investigate the convergence and consistency properties of an Invariant-Extended Kalman Filter (RI-EKF) based Simultaneous Localization and Mapping (SLAM) algorithm. Basic convergence properties of this algorithm are proven. These proofs do not require the restrictive assumption that the Jacobians of the motion and observation models need to be evaluated at the ground truth. It is also shown that the output of RI-EKF is invariant under any stochastic rigid body transformation in contrast to $\mathbb{SO}(3)$ based EKF SLAM algorithm ($\mathbb{SO}(3)$-EKF) that is only invariant…
Recent progress in object pose prediction provides a promising path for robots to build object-level scene representations during navigation. However, as we deploy a robot in novel environments, the out-of-distribution data can degrade the prediction performance. To mitigate the domain gap, we can potentially perform self-training in the target domain, using predictions on robot-captured images as pseudo labels to fine-tune the object pose estimator. Unfortunately, the pose predictions are typically outlier-corrupted, and it is hard to quantify their uncertainties, which can result in low-qual…
FUSE: A Framework for Unified State Estimation in Vehicular and Robotic SLAM Systems
State Key Lab of Intelligent Green Vehicle and Mobility, Tsinghua University（智能绿色车辆与移动国家重点实验室，清华大学）；School of VM and College of AI, Tsinghua University（车辆学院与人工智能学院，清华大学）；SunRisingAI Ltd.（SunRisingAI有限公司）；China Intelligent and Connected Vehicles (Beijing) Research Institute Co., Ltd.（中国汽车智能互联车辆（北京）研究院有限公司）
Simultaneous localization and mapping (SLAM) are crucial for autonomous robots (e.g., self-driving cars, autonomous drones), 3D mapping systems, and AR/VR applications. This work proposed a novel LiDAR-inertial-visual fusion framework termed R$^3$LIVE++ to achieve robust and accurate state estimation while simultaneously reconstructing the radiance map on the fly. R$^3$LIVE++ consists of a LiDAR-inertial odometry (LIO) and a visual-inertial odometry (VIO), both running in real-time. The LIO subsystem utilizes the measurements from a LiDAR for reconstructing the geometric structure (i.e., the p…
GeVI-SLAM: Gravity-Enhanced Stereo Visua Inertial SLAM for Underwater Robots
School of Data Science, Chinese University of Hong Kong, Shenzhen, China（数据科学学院，香港中文大学（深圳））；School of Life Sciences, Chinese University of Hong Kong, China（生命科学学院，香港中文大学）
Stevens Institute of Technology（史蒂文斯理工学院）；Massachusetts Institute of Technology（麻省理工学院）
The ability to infer map variables and estimate pose is crucial to the operation of autonomous mobile robots. In most cases the shared dependency between these variables is modeled through a multivariate Gaussian distribution, but there are many situations where that assumption is unrealistic. Our paper shows how it is possible to relax this assumption and perform simultaneous localization and mapping (SLAM) with a larger class of distributions, whose multivariate dependency is represented with a copula model. We integrate the distribution model with copulas into a Sequential Monte Carlo estim…
École Polytechnique de Montréal（蒙特利尔理工学院）；Massachusetts Institute of Technology（麻省理工学院）
Perceptual aliasing is one of the main causes of failure for Simultaneous Localization and Mapping (SLAM) systems operating in the wild. Perceptual aliasing is the phenomenon where different places generate a similar visual (or, in general, perceptual) footprint. This causes spurious measurements to be fed to the SLAM estimator, which typically results in incorrect localization and mapping results. The problem is exacerbated by the fact that those outliers are highly correlated, in the sense that perceptual aliasing creates a large number of mutually-consistent outliers. Another issue stems fr…
Fast and Accurate Deep Loop Closing and Relocalization for Reliable LiDAR SLAM
College of Intelligence Science and Technology, National University of Defense Technology（国防科技大学智能科学与技术学院）；National Innovation Institution of Defense Technology（国防科技国家创新研究院）
本文提出统一框架 LCR-Net，通过特征提取和位姿感知注意力机制精确估计 LiDAR 扫描相似度与 6-DoF 位姿，实现快速准确的深度闭环与重定位，并集成至 SLAM 系统提升在线可靠性。
Spatiotemporal Calibration and Ground Truth Estimation for High-Precision SLAM Benchmarking in Extended Reality
Advanced Display and Sensing Research Center, Yongjiang Laboratory, Ningbo, China（先进显示与感知研究中心，永江实验室，宁波，中国）；Shanghai Institute of Technical Physics, Chinese Academy of Sciences, Shanghai, China（上海技术物理研究所，中国科学院，上海，中国）
Wuhan University（武汉大学）；ETH Zurich（苏黎世联邦理工学院）；Washington University, St. Louis（圣路易斯华盛顿大学）
State estimation problems without absolute position measurements routinely arise in navigation of unmanned aerial vehicles, autonomous ground vehicles, etc., whose proper operation relies on accurate state estimates and reliable covariances. Unaware of absolute positions, these problems have immanent unobservable directions. Traditional causal estimators, however, usually gain spurious information on the unobservable directions, leading to over-confident covariance inconsistent with actual estimator errors. The consistency problem of fixed-lag smoothers (FLSs) has only been attacked by the fir…
This paper presents a novel tightly-coupled monocular visual-inertial Simultaneous Localization and Mapping algorithm, which provides accurate and robust localization within the globally consistent map in real time on a standard CPU. This is achieved by firstly performing the visual-inertial extended kalman filter(EKF) to provide motion estimate at a high rate. However the filter becomes inconsistent due to the well known linearization issues. So we perform a keyframe-based visual-inertial bundle adjustment to improve the consistency and accuracy of the system. In addition, a loop closure dete…
Thompson Rivers University（汤普森河大学）；The Hong Kong Polytechnic University（香港理工大学）
Simultaneous Localization and Mapping (SLAM) is a process of concurrent estimation of the vehicle's pose and feature locations with respect to a frame of reference. This paper proposes a computationally cheap geometric nonlinear SLAM filter algorithm structured to mimic the nonlinear motion dynamics of the true SLAM problem posed on the matrix Lie group of $\mathbb{SLAM}_{n}\left(3\right)$. The nonlinear filter on manifold is proposed in continuous form and it utilizes available measurements obtained from group velocity vectors, feature measurements and an inertial measurement unit (IMU). The …
Technical University of Munich（慕尼黑工业大学）；Imperial College London（伦敦帝国学院）
Robotic applications are continuously striving towards higher levels of autonomy. To achieve that goal, a highly robust and accurate state estimation is indispensable. Combining visual and inertial sensor modalities has proven to yield accurate and locally consistent results in short-term applications. Unfortunately, visual-inertial state estimators suffer from the accumulation of drift for long-term trajectories. To eliminate this drift, global measurements can be fused into the state estimation pipeline. The most known and widely available source of global measurements is the Global Position…
A reliable pose estimator robust to environmental disturbances is desirable for mobile robots. To this end, inertial measurement units (IMUs) play an important role because they can perceive the full motion state of the vehicle independently. However, it suffers from accumulative error due to inherent noise and bias instability, especially for low-cost sensors. In our previous studies on Wheel-INS \cite{niu2021, wu2021}, we proposed to limit the error drift of the pure inertial navigation system (INS) by mounting an IMU to the wheel of the robot to take advantage of rotation modulation. Howeve…
Oxford Robotics Institute, University of Oxford（牛津大学牛津机器人研究所）
In this paper, we present a factor-graph LiDAR-SLAM system which incorporates a state-of-the-art deeply learned feature-based loop closure detector to enable a legged robot to localize and map in industrial environments. These facilities can be badly lit and comprised of indistinct metallic structures, thus our system uses only LiDAR sensing and was developed to run on the quadruped robot's navigation PC. Point clouds are accumulated using an inertial-kinematic state estimator before being aligned using ICP registration. To close loops we use a loop proposal mechanism which matches individual …
OKVIS2-X: Open Keyframe-based Visual-Inertial SLAM Configurable with Dense Depth or LiDAR, and GNSS
Mobile Robotics Lab, School of Computation, Information and Technology (CIT) at the Technical University of Munich (TUM)（慕尼黑技术大学移动机器人实验室，计算信息与技术学院）；Munich Institute of Robotics and Machine Intelligence (MIRMI)（慕尼黑机器人与机器智能研究所）；Mobile Robotics Lab, ETH Zurich（苏黎世联邦理工学院移动机器人实验室）
VIMS: A Visual-Inertial-Magnetic-Sonar SLAM System in Underwater Environments
Key Laboratory of Ocean Observation-Imaging Testbed of Zhejiang Province, Zhejiang University（浙江省海洋观测-成像测试床重点实验室，浙江大学）；Interdisciplinary Student Training Platform for Marine areas, Zhejiang University（海洋领域交叉人才培养平台，浙江大学）；Department of Electronic and Computer Engineering, The Hong Kong University of Science and Technology（电子与计算机工程系，香港科技大学）；The Institute of Deep-Sea Science and Engineering, Chinese Academy of Sciences（深海科学与工程研究院，中国科学院）
School of Artificial Intelligence and Automation, Huazhong University of Science and Technology（华中科技大学人工智能与自动化学院）
The visual SLAM method is widely used for self-localization and mapping in complex environments. Visual-inertia SLAM, which combines a camera with IMU, can significantly improve the robustness and enable scale weak-visibility, whereas monocular visual SLAM is scale-invisible. For ground mobile robots, the introduction of a wheel speed sensor can solve the scale weak-visible problem and improve the robustness under abnormal conditions. In this thesis, a multi-sensor fusion SLAM algorithm using monocular vision, inertia, and wheel speed measurements is proposed. The sensor measurements are combi…
A Consistency-Improved LiDAR-Inertial Bundle Adjustment
Aerospace Information Research Institute, Chinese Academy of Sciences（中国科学院航空信息研究所）；University of Chinese Academy of Sciences（中国科学院大学）；Aerospace Information Technology University（航空信息科技大学）；QiLu Aerospace Information Research Institute, Chinese Academy of Sciences（齐鲁航空信息研究所）；School of Electronic, Electrical and Communication Engineering（电子电气与通信工程学院）
本文提出改进一致性的 LiDAR-惯性束调整方法，通过定制参数化和估计器提升 SLAM 系统的准确性和可观测性。
A robot operating in a household makes observations of multiple objects as it moves around over the course of days or weeks. The objects may be moved by inhabitants, but not completely at random. The robot may be called upon later to retrieve objects and will need a long-term object-based memory in order to know how to find them. Existing work in semantic slam does not attempt to capture the dynamics of object movement. In this paper, we combine some aspects of classic techniques for data-association filtering with modern attention-based neural networks to construct object-based memory systems…
As autonomous systems increasingly rely on onboard sensing for localization and perception, the parallel tasks of motion planning and state estimation become more strongly coupled. This coupling is well-captured by augmenting the planning objective with a posterior-covariance penalty -- however, prediction of the estimator covariance is challenging when the observation model depends on unknown landmarks, as is the case in Simultaneous Localization and Mapping (SLAM). This paper addresses these challenges in the case of landmark- and SLAM-based estimators, enabling efficient prediction (and ult…
