Scan undistortion is a key module for LiDAR odometry in high dynamic environment with high rotation and translation speed. The existing line of studies mostly focuses on one pass undistortion, which means undistortion for each point is conducted only once in the whole LiDAR-IMU odometry pipeline. In this paper, we propose an optimization based tightly coupled LiDAR-IMU odometry addressing iterated point-level undistortion. By jointly minimizing the cost derived from LiDAR and IMU measurements, our LiDAR-IMU odometry method performs more accurate and robust in high dynamic environment. Besides, the method characters good computation efficiency by limiting the quantity of parameters.
Vibration-aware Lidar-Inertial Odometry based on Point-wise Post-Undistortion Uncertainty
State Key Laboratory of Intelligent Manufacturing Equipment and Technology, School of Mechanical Science and Engineering, Huazhong University of Science and Technology（智能制造装备与技术国家重点实验室，机械科学与工程学院，华中科技大学）
RAM-LAB（机器人与自动化实验室）；Hong Kong University of Science and Technology（香港科技大学）
Ego-motion estimation is a fundamental requirement for most mobile robotic applications. By sensor fusion, we can compensate the deficiencies of stand-alone sensors and provide more reliable estimations. We introduce a tightly coupled lidar-IMU fusion method in this paper. By jointly minimizing the cost derived from lidar and IMU measurements, the lidar-IMU odometry (LIO) can perform well with acceptable drift after long-term experiment, even in challenging cases where the lidar measurements can be degraded. Besides, to obtain more reliable estimations of the lidar poses, a rotation-constraine…
The paper presents a direct visual-inertial odometry system. In particular, a tightly coupled nonlinear optimization based method is proposed by integrating the recent advances in direct dense tracking and Inertial Measurement Unit (IMU) pre-integration, and a factor graph optimization is adapted to estimate the pose of the camera and rebuild a semi-dense map. Two sliding windows are maintained in the proposed approach. The first one, based on Direct Sparse Odometry (DSO), is to estimate the depths of candidate points for mapping and dense visual tracking. In the second one, measurements from …
We present an unsupervised deep neural network approach to the fusion of RGB-D imagery with inertial measurements for absolute trajectory estimation. Our network, dubbed the Visual-Inertial-Odometry Learner (VIOLearner), learns to perform visual-inertial odometry (VIO) without inertial measurement unit (IMU) intrinsic parameters (corresponding to gyroscope and accelerometer bias or white noise) or the extrinsic calibration between an IMU and camera. The network learns to integrate IMU measurements and generate hypothesis trajectories which are then corrected online according to the Jacobians o…
Visual odometry and SLAM methods have a large variety of applications in domains such as augmented reality or robotics. Complementing vision sensors with inertial measurements tremendously improves tracking accuracy and robustness, and thus has spawned large interest in the development of visual-inertial (VI) odometry approaches. In this paper, we propose the TUM VI benchmark, a novel dataset with a diverse set of sequences in different scenes for evaluating VI odometry. It provides camera images with 1024x1024 resolution at 20 Hz, high dynamic range and photometric calibration. An IMU measure…
Nanjing University of Science and Technology（南京理工大学）；PCA Lab（PCA实验室）
Extensive research efforts have been dedicated to deep learning based odometry. Nonetheless, few efforts are made on the unsupervised deep lidar odometry. In this paper, we design a novel framework for unsupervised lidar odometry with the IMU, which is never used in other deep methods. First, a pair of siamese LSTMs are used to obtain the initial pose from the linear acceleration and angular velocity of IMU. With the initial pose, we perform the rigid transform on the current frame and align it closer to the last frame. Then, we extract vertex and normal features from the transformed point clo…
This paper introduces the united monocular-stereo features into a visual-inertial tightly coupled odometry (UMS-VINS) for robust pose estimation. UMS-VINS requires two cameras and a low-cost inertial measurement unit (IMU). The UMS-VINS is an evolution of VINS-FUSION, which modifies the VINS-FUSION from the following three perspectives. 1) UMS-VINS extracts and tracks features from the sub-pixel plane to achieve better positions of the features. 2) UMS-VINS introduces additional 2-dimensional features from the left and/or right cameras. 3) If the visual initialization fails, the IMU propagatio…
Introducing object-level semantic information into simultaneous localization and mapping (SLAM) system is critical. It not only improves the performance but also enables tasks specified in terms of meaningful objects. This work presents OrcVIO, for visual-inertial odometry tightly coupled with tracking and optimization over structured object models. OrcVIO differentiates through semantic feature and bounding-box reprojection errors to perform batch optimization over the pose and shape of objects. The estimated object states aid in real-time incremental optimization over the IMU-camera states. …
Zhejiang University（浙江大学）；University of Delaware（特拉华大学）
This paper presents a tightly-coupled multi-sensor fusion algorithm termed LiDAR-inertial-camera fusion (LIC-Fusion), which efficiently fuses IMU measurements, sparse visual features, and extracted LiDAR points. In particular, the proposed LIC-Fusion performs online spatial and temporal sensor calibration between all three asynchronous sensors, in order to compensate for possible calibration variations. The key contribution is the optimal (up to linearization errors) multi-modal sensor fusion of detected and tracked sparse edge/surf feature points from LiDAR scans within an efficient MSCKF-bas…
LiDAR-Inertial Odometry Based on Extended Kalman Filter
Institute of Innovation for Future Society, Nagoya University（名古屋大学未来社会创新研究所）；LOCT Co., Ltd.（LOCT有限公司）；Graduate School of Science and Technology, University of Tsukuba（筑波大学理工学研究科）
For most LiDAR-inertial odometry, accurate initial states, including temporal offset and extrinsic transformation between LiDAR and 6-axis IMUs, play a significant role and are often considered as prerequisites. However, such information may not be always available in customized LiDAR-inertial systems. In this paper, we propose LI-Init: a full and real-time LiDAR-inertial system initialization process that calibrates the temporal offset and extrinsic parameter between LiDARs and IMUs, and also the gravity vector and IMU bias by aligning the state estimated from LiDAR measurements with that mea…
We present a dead reckoning strategy for increased resilience to position estimation failures on multirotors, using only data from a low-cost IMU and novel, bio-inspired airflow sensors. The goal is challenging, since low-cost IMUs are subject to large noise and drift, while 3D airflow sensing is made difficult by the interference caused by the propellers and by the wind. Our approach relies on a deep-learning strategy to interpret the measurements of the bio-inspired sensors, a map of the wind speed to compensate for position-dependent wind, and a filter to fuse the information and generate a…
Massachusetts Institute of Technology（麻省理工学院）；Stevens Institute of Technology（斯蒂文斯理工学院）
We propose a framework for tightly-coupled lidar inertial odometry via smoothing and mapping, LIO-SAM, that achieves highly accurate, real-time mobile robot trajectory estimation and map-building. LIO-SAM formulates lidar-inertial odometry atop a factor graph, allowing a multitude of relative and absolute measurements, including loop closures, to be incorporated from different sources as factors into the system. The estimated motion from inertial measurement unit (IMU) pre-integration de-skews point clouds and produces an initial guess for lidar odometry optimization. The obtained lidar odomet…
Adaptive-LIO: Enhancing Robustness and Precision through Environmental Adaptation in LiDAR Inertial Odometry
State Key Laboratory of Robotics and Systems（机器人学与系统国家重点实验室）；Harbin Institute of Technology（哈尔滨工业大学）；Hangzhou Qisheng Intelligent Techology Co. Ltd.（杭州齐盛智能科技有限公司）；School of Mechatronic Engineering, China University of Mining and Technology（中国矿业大学机电工程学院）；School of Mechanical and Electrical Engineering, University of Electronic Science And Technology（电子科技大学机械与电气工程学院）；School of Electrical and Electronic Engineering, Nanyang Technological University（南洋理工大学电气与电子工程学院）
University of Portsmouth（朴茨茅斯大学）；Plymouth University（普利茅斯大学）
We tackle the problem of localizing an autonomous sea-surface vehicle in river estuarine areas using monocular camera and angular velocity input from an inertial sensor. Our method is challenged by two prominent drawbacks associated with the environment, which are typically not present in standard visual simultaneous localization and mapping (SLAM) applications on land (or air): a) Scene depth varies significantly (from a few meters to several kilometers) and, b) In conjunction to the latter, there exists no ground plane to provide features with enough disparity based on which to reliably dete…
Visual-Inertial Odometry (VIO) utilizes an Inertial Measurement Unit (IMU) to overcome the limitations of Visual Odometry (VO). However, the VIO for vehicles in large-scale outdoor environments still has some difficulties in estimating forward motion with distant features. To solve these difficulties, we propose a robust VIO method based on the analysis of feature confidence in forward motion estimation using an IMU. We first formulate the VIO problem by using effective trifocal tensor geometry. Then, we infer the feature confidence by using the motion information obtained from an IMU and inco…
GNSS Research Center, Wuhan University（武汉大学卫星导航定位技术研究中心）
We present RailLoMer in this article, to achieve real-time accurate and robust odometry and mapping for rail vehicles. RailLoMer receives measurements from two LiDARs, an IMU, train odometer, and a global navigation satellite system (GNSS) receiver. As frontend, the estimated motion from IMU/odometer preintegration de-skews the denoised point clouds and produces initial guess for frame-to-frame LiDAR odometry. As backend, a sliding window based factor graph is formulated to jointly optimize multi-modal information. In addition, we leverage the plane constraints from extracted rail tracks and t…
Institute of Artificial Intelligence, Huazhong University of Science and Technology（华中科技大学人工智能研究院）；Electronic Information and Communications, Huazhong University of Science and Technology（华中科技大学电子信息与通信学院）
In this paper, we propose a tightly-coupled, multi-modal simultaneous localization and mapping (SLAM) framework, integrating an extensive set of sensors: IMU, cameras, multiple lidars, and Ultra-wideband (UWB) range measurements, hence referred to as VIRAL (visual-inertial-ranging-lidar) SLAM. To achieve such a comprehensive sensor fusion system, one has to tackle several challenges such as data synchronization, multi-threading programming, bundle adjustment (BA), and conflicting coordinate frames between UWB and the onboard sensors, so as to ensure real-time localization and smooth updates in…
We propose an accurate and robust multi-modal sensor fusion framework, MetroLoc, towards one of the most extreme scenarios, the large-scale metro vehicle localization and mapping. MetroLoc is built atop an IMU-centric state estimator that tightly couples light detection and ranging (LiDAR), visual, and inertial information with the convenience of loosely coupled methods. The proposed framework is composed of three submodules: IMU odometry, LiDAR-inertial odometry (LIO), and Visual-inertial odometry (VIO). The IMU is treated as the primary sensor, which achieves the observations from LIO and VI…
Institute for Infocomm Research, A-STAR（新加坡科技研究局信息通信研究院）；Nanyang Technological University（南洋理工大学）
Light detection and ranging (LiDAR) on a moving agent could suffer from motion distortion due to simultaneous rotation of the LiDAR and fast movement of the agent. An accurate piecewise linear de skewing algorithm is proposed to correct the motion distortions for LiDAR inertial odometry (LIO) using high frequency motion information provided by an Inertial Measurement Unit (IMU). Experimental results show that the proposed algorithm can be adopted to improve the performance of existing LIO algorithms especially in cases of fast movement.
ESVO2: Direct Visual-Inertial Odometry with Stereo Event Cameras
School of Robotics, Hunan University（湖南大学机器人学院）；Neuromorphic Automation and Intelligence Lab (NAIL)（神经形态自动化与智能实验室）；Department of Electronic and Computer Engineering（电子与计算机工程系）；Hong Kong University of Science and Technology（香港科学与技术大学）；TU Berlin（柏林工业大学）；Science of Intelligence Excellence Cluster（智能科学卓越集群）；Robotics Institute Germany（德国机器人研究所）；Einstein Center Digital Future（爱因斯坦中心数字未来）
National Institute of Advanced Industrial Science and Technology（国立先进工业科学技术研究所）
This paper presents a real-time 3D mapping framework based on global matching cost minimization and LiDAR-IMU tight coupling. The proposed framework comprises a preprocessing module and three estimation modules: odometry estimation, local mapping, and global mapping, which are all based on the tight coupling of the GPU-accelerated voxelized GICP matching cost factor and the IMU preintegration factor. The odometry estimation module employs a keyframe-based fixed-lag smoothing approach for efficient and low-drift trajectory estimation, with a bounded computation cost. The global mapping module c…
Karlsruhe Institute of Technology (KIT)（卡尔斯鲁厄理工学院）
We present a novel tightly-coupled LiDAR-inertial odometry and mapping scheme for both solid-state and mechanical LiDARs. As frontend, a feature-based lightweight LiDAR odometry provides fast motion estimates for adaptive keyframe selection. As backend, a hierarchical keyframe-based sliding window optimization is performed through marginalization for directly fusing IMU and LiDAR measurements. For the Livox Horizon, a newly released solid-state LiDAR, a novel feature extraction method is proposed to handle its irregular scan pattern during preprocessing. LiLi-OM (Livox LiDAR-inertial odometry …
Nanyang Technological University（南洋理工大学）；Institute for Infocomm Research, A-STAR（新加坡科技研究局资讯通信研究院）
Employing an inertial measurement unit (IMU) as an additional sensor can dramatically improve both reliability and accuracy of visual/Lidar odometry (VO/LO). Different IMU integration models are introduced using different assumptions on the linear acceleration from the IMU. In this paper, a novel IMU integration model is proposed by using switched linear systems. The proposed approach assumes that both the linear acceleration and the angular velocity in the body frame are constant between two consecutive IMU measurements. This is more realistic in real world situation compared to existing appr…
AKF-LIO: LiDAR-Inertial Odometry with Gaussian Map by Adaptive Kalman Filter
The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；International Digital Economy Academy（国际数字经济学院）；The Hong Kong University of Science and Technology（香港科技大学）；Southern University of Science and Technology（南方科技大学）
Integrating multiple LiDAR sensors can significantly enhance a robot's perception of the environment, enabling it to capture adequate measurements for simultaneous localization and mapping (SLAM). Indeed, solid-state LiDARs can bring in high resolution at a low cost to traditional spinning LiDARs in robotic applications. However, their reduced field of view (FoV) limits performance, particularly indoors. In this paper, we propose a tightly-coupled multi-modal multi-LiDAR-inertial SLAM system for surveying and mapping tasks. By taking advantage of both solid-state and spinnings LiDARs, and buil…
The ubiquitous planes and structural consistency are the most apparent features of indoor multi-story Buildings compared with outdoor environments. In this paper, we propose a tightly coupled LiDAR-Inertial 3D SLAM framework with plane features for the multi-story building. The framework we proposed is mainly composed of three parts: tightly coupled LiDAR-Inertial odometry, extraction of representative planes of the structure, and factor graph optimization. By building a local map and inertial measurement unit (IMU) pre-integration, we get LiDAR scan-to-local-map matching and IMU measurements,…
This paper presents a computationally efficient and robust LiDAR-inertial odometry framework. We fuse LiDAR feature points with IMU data using a tightly-coupled iterated extended Kalman filter to allow robust navigation in fast-motion, noisy or cluttered environments where degeneration occurs. To lower the computation load in the presence of large number of measurements, we present a new formula to compute the Kalman gain. The new formula has computation load depending on the state dimension instead of the measurement dimension. The proposed method and its implementation are tested in various …
Shenzhen Unity-Drive Innovation Technology Co. Ltd.（深圳一清创新科技有限公司）；Hong Kong University of Science and Technology（香港科技大学）；Hangzhou ATG Intelligent Technology Co. Ltd.（杭州ATG智能科技有限公司）
Using different sensors in an autonomous vehicle (AV) can provide multiple constraints to optimize AV location estimation. In this paper, we present a low-cost GPS-assisted LiDAR state estimation system for AVs. Firstly, we utilize LiDAR to obtain highly precise 3D geometry data. Next, we use an inertial measurement unit (IMU) to correct point cloud misalignment caused by incorrect place recognition. The estimated LiDAR odometry and IMU measurement are then jointly optimized. We use a lost-cost GPS instead of a real-time kinematic (RTK) module to refine the estimated LiDAR-inertial odometry. O…
MINES ParisTech, PSL Research University（巴黎高等矿业学院，巴黎文理研究大学）；Safran Tech, Groupe Safran（赛峰集团赛峰科技）
In this paper we propose a novel accurate method for dead-reckoning of wheeled vehicles based only on an Inertial Measurement Unit (IMU). In the context of intelligent vehicles, robust and accurate dead-reckoning based on the IMU may prove useful to correlate feeds from imaging sensors, to safely navigate through obstructions, or for safe emergency stops in the extreme case of exteroceptive sensors failure. The key components of the method are the Kalman filter and the use of deep neural networks to dynamically adapt the noise parameters of the filter. The method is tested on the KITTI odometr…
A Numerically-Robust ROS 2 Port of iG-LIO: Diagnosing and Fixing Toolchain-Induced Failures in Incremental GICP LiDAR-Inertial Odometry
Robotics Institute (RI), Carnegie Mellon University (CMU)（卡内基梅隆大学机器人研究所）；Institute of Systems and Robotics (ISR), University of Coimbra (UC)（科英布拉大学系统与机器人研究所）
Zhejiang University（浙江大学）；Karlsruhe Institute of Technology（卡尔斯鲁厄理工学院）；Huzhou Institute of Zhejiang University（浙江大学湖州研究院）
Visual-inertial-odometry has attracted extensive attention in the field of autonomous driving and robotics. The size of Field of View (FoV) plays an important role in Visual-Odometry (VO) and Visual-Inertial-Odometry (VIO), as a large FoV enables to perceive a wide range of surrounding scene elements and features. However, when the field of the camera reaches the negative half plane, one cannot simply use [u,v,1]^T to represent the image feature points anymore. To tackle this issue, we propose LF-VIO, a real-time VIO framework for cameras with extremely large FoV. We leverage a three-dimension…
Localization is an essential technique in mobile robotics. In a complex environment, it is necessary to fuse different localization modules to obtain more robust results, in which the error model plays a paramount role. However, exteroceptive sensor-based odometries (ESOs), such as LiDAR/visual odometry, often deliver results with scene-related error, which is difficult to model accurately. To address this problem, this research designs a scene-aware error model for ESO, based on which a multimodal localization fusion framework is developed. In addition, an end-to-end learning method is propos…
