Modern autonomous systems typically use several sensors for perception. For best performance, accurate and reliable extrinsic calibration is necessary. In this research, we propose a reliable technique for the extrinsic calibration of several lidars on a vehicle without the need for odometry estimation or fiducial markers. First, our method generates an initial guess of the extrinsics by matching the raw signals of IMUs co-located with each lidar. This initial guess is then used in ICP and point cloud feature matching which refines and verifies this estimate. Furthermore, we can use observability criteria to choose a subset of the IMU measurements that have the highest mutual information -- rather than comparing all the readings. We have successfully validated our methodology using data gathered from Scania test vehicles.
Institute of Cyber-Systems and Control, Zhejiang University（浙江大学工业控制研究所）
Sensor calibration is the fundamental block for a multi-sensor fusion system. This paper presents an accurate and repeatable LiDAR-IMU calibration method (termed LI-Calib), to calibrate the 6-DOF extrinsic transformation between the 3D LiDAR and the Inertial Measurement Unit (IMU). % Regarding the high data capture rate for LiDAR and IMU sensors, LI-Calib adopts a continuous-time trajectory formulation based on B-Spline, which is more suitable for fusing high-rate or asynchronous measurements than discrete-time based approaches. % Additionally, LI-Calib decomposes the space into cells and iden…
In this work we present a novel method to jointly calibrate a sensor suite consisting a 3D-LiDAR, Inertial Measurement Unit (IMU) and Camera under an Extended Kalman Filter (EKF) framework. We exploit pairwise constraints between the 3 sensor pairs to perform EKF update and experimentally demonstrate the superior performance obtained with joint calibration as against individual sensor pair calibration.
The University of Tokyo（东京大学）；Southeast University（东南大学）
Recently, multi-sensors fusion has achieved significant progress in the field of automobility to improve navigation and position performance. As the prerequisite of the fusion algorithm, the demand for the extrinsic calibration of multi-sensors is growing. To calculate the extrinsic parameter, many researches have been dedicated to the two-step method, which integrates the respective calibration in pairs. It is inefficient and incompact because of losing sight of the constrain of all sensors. With regard to remove this burden, an optimization-based IMU/Lidar/Camera co-calibration method is pro…
Texas A&M University（得克萨斯农工大学）；Ford Motor Company（福特汽车公司）
This work presents a novel target-free extrinsic calibration algorithm for a 3D Lidar and an IMU pair using an Extended Kalman Filter (EKF) which exploits the \textit{motion based calibration constraint} for state update. The steps include, data collection by motion excitation of the Lidar Inertial Sensor suite along all degrees of freedom, determination of the inter sensor rotation by using rotational component of the aforementioned \textit{motion based calibration constraint} in a least squares optimization framework, and finally, the determination of inter sensor translation using the \text…
For most LiDAR-inertial odometry, accurate initial states, including temporal offset and extrinsic transformation between LiDAR and 6-axis IMUs, play a significant role and are often considered as prerequisites. However, such information may not be always available in customized LiDAR-inertial systems. In this paper, we propose LI-Init: a full and real-time LiDAR-inertial system initialization process that calibrates the temporal offset and extrinsic parameter between LiDARs and IMUs, and also the gravity vector and IMU bias by aligning the state estimated from LiDAR measurements with that mea…
We propose Super Odometry, a high-precision multi-modal sensor fusion framework, providing a simple but effective way to fuse multiple sensors such as LiDAR, camera, and IMU sensors and achieve robust state estimation in perceptually-degraded environments. Different from traditional sensor-fusion methods, Super Odometry employs an IMU-centric data processing pipeline, which combines the advantages of loosely coupled methods with tightly coupled methods and recovers motion in a coarse-to-fine manner. The proposed framework is composed of three parts: IMU odometry, visual-inertial odometry, and …
Scan undistortion is a key module for LiDAR odometry in high dynamic environment with high rotation and translation speed. The existing line of studies mostly focuses on one pass undistortion, which means undistortion for each point is conducted only once in the whole LiDAR-IMU odometry pipeline. In this paper, we propose an optimization based tightly coupled LiDAR-IMU odometry addressing iterated point-level undistortion. By jointly minimizing the cost derived from LiDAR and IMU measurements, our LiDAR-IMU odometry method performs more accurate and robust in high dynamic environment. Besides,…
Zhejiang University（浙江大学）；University of Delaware（特拉华大学）；Huzhou Institute of Zhejiang University（浙江大学湖州研究院）
Accurate and reliable sensor calibration is essential to fuse LiDAR and inertial measurements, which are usually available in robotic applications. In this paper, we propose a novel LiDAR-IMU calibration method within the continuous-time batch-optimization framework, where the intrinsics of both sensors and the spatial-temporal extrinsics between sensors are calibrated without using calibration infrastructure such as fiducial tags. Compared to discrete-time approaches, the continuous-time formulation has natural advantages for fusing high rate measurements from LiDAR and IMU sensors. To improv…
Faculty of Information Technology, Brno University of Technology（布尔诺理工大学信息技术学院）
We introduce a novel method for odometry estimation using convolutional neural networks from 3D LiDAR scans. The original sparse data are encoded into 2D matrices for the training of proposed networks and for the prediction. Our networks show significantly better precision in the estimation of translational motion parameters comparing with state of the art method LOAM, while achieving real-time performance. Together with IMU support, high quality odometry estimation and LiDAR data registration is realized. Moreover, we propose alternative CNNs trained for the prediction of rotational motion pa…
In this paper, we propose a tightly-coupled, multi-modal simultaneous localization and mapping (SLAM) framework, integrating an extensive set of sensors: IMU, cameras, multiple lidars, and Ultra-wideband (UWB) range measurements, hence referred to as VIRAL (visual-inertial-ranging-lidar) SLAM. To achieve such a comprehensive sensor fusion system, one has to tackle several challenges such as data synchronization, multi-threading programming, bundle adjustment (BA), and conflicting coordinate frames between UWB and the onboard sensors, so as to ensure real-time localization and smooth updates in…
GNSS Research Center, Wuhan University（武汉大学卫星导航定位技术研究中心）
We present RailLoMer in this article, to achieve real-time accurate and robust odometry and mapping for rail vehicles. RailLoMer receives measurements from two LiDARs, an IMU, train odometer, and a global navigation satellite system (GNSS) receiver. As frontend, the estimated motion from IMU/odometer preintegration de-skews the denoised point clouds and produces initial guess for frame-to-frame LiDAR odometry. As backend, a sliding window based factor graph is formulated to jointly optimize multi-modal information. In addition, we leverage the plane constraints from extracted rail tracks and t…
RAM-LAB（机器人与自动化实验室）；Hong Kong University of Science and Technology（香港科技大学）
Ego-motion estimation is a fundamental requirement for most mobile robotic applications. By sensor fusion, we can compensate the deficiencies of stand-alone sensors and provide more reliable estimations. We introduce a tightly coupled lidar-IMU fusion method in this paper. By jointly minimizing the cost derived from lidar and IMU measurements, the lidar-IMU odometry (LIO) can perform well with acceptable drift after long-term experiment, even in challenging cases where the lidar measurements can be degraded. Besides, to obtain more reliable estimations of the lidar poses, a rotation-constraine…
Tightly-Coupled LiDAR-IMU-Wheel Odometry with an Online Neural Kinematic Model Learning via Factor Graph Optimization
The Space Robotics Lab. in the Department of Aerospace Engineering, Graduate School of Engineering, Tohoku University（东京大学工学研究生院航空航天工程系空间机器人实验室）；National Institute of Advanced Industrial Science and Technology (AIST)（国家先进工业科学与技术研究院）
National Institute of Advanced Industrial Science and Technology（国立先进工业科学技术研究所）
This paper presents a real-time 3D mapping framework based on global matching cost minimization and LiDAR-IMU tight coupling. The proposed framework comprises a preprocessing module and three estimation modules: odometry estimation, local mapping, and global mapping, which are all based on the tight coupling of the GPU-accelerated voxelized GICP matching cost factor and the IMU preintegration factor. The odometry estimation module employs a keyframe-based fixed-lag smoothing approach for efficient and low-drift trajectory estimation, with a bounded computation cost. The global mapping module c…
LiDAR-Inertial Odometry Based on Extended Kalman Filter
Institute of Innovation for Future Society, Nagoya University（名古屋大学未来社会创新研究所）；LOCT Co., Ltd.（LOCT有限公司）；Graduate School of Science and Technology, University of Tsukuba（筑波大学理工学研究科）
Zhejiang University（浙江大学）；University of Delaware（特拉华大学）
This paper presents a tightly-coupled multi-sensor fusion algorithm termed LiDAR-inertial-camera fusion (LIC-Fusion), which efficiently fuses IMU measurements, sparse visual features, and extracted LiDAR points. In particular, the proposed LIC-Fusion performs online spatial and temporal sensor calibration between all three asynchronous sensors, in order to compensate for possible calibration variations. The key contribution is the optimal (up to linearization errors) multi-modal sensor fusion of detected and tracked sparse edge/surf feature points from LiDAR scans within an efficient MSCKF-bas…
FF-LINS: A Consistent Frame-to-Frame Solid-State-LiDAR-Inertial State Estimator
GNSS Research Center, Wuhan University（武汉大学GNSS研究中心）；Hubei Luojia Laboratory（湖北珞珈实验室）；Artificial Intelligence Institute of Wuhan University（武汉大学人工智能研究院）
This paper presents Lidar-based Simultaneous Localization and Mapping (SLAM) for autonomous driving vehicles. Fusing data from landmark sensors and a strap-down Inertial Measurement Unit (IMU) in an adaptive Kalman filter (KF) plus the observability of the system are investigated. In addition to the vehicle's states and landmark positions, a self-tuning filter estimates the IMU calibration parameters as well as the covariance of the measurement noise. The discrete-time covariance matrix of the process noise, the state transition matrix, and the observation sensitivity matrix are derived in clo…
School of Artificial Intelligence and Automation, Huazhong University of Science and Technology（华中科技大学人工智能与自动化学院）；School of Automation, SouthEast University（东南大学自动化学院）
Mobile robot positioning, mapping, and navigation systems generally employ an inertial measurement unit (IMU) to obtain the acceleration and angular velocity of the robot. However, errors in the internal and external parameters of an IMU arising from defective calibration directly affect the accuracy of robot positioning and pose estimation. While this issue has been addressed by the mature internal reference calibration methods available for IMUs, external reference calibration methods between the IMU and the chassis of a mobile robot are lacking. This study addresses this issue by proposing …
The Hong Kong University of Science and Technology（香港科技大学）
Pose estimation purely based on 3D point-cloud could suffer from degradation, e.g. scan blocks or scans in repetitive environments. To deal with this problem, we propose an approach for fusing 3D spinning LiDAR and IMU to estimate the ego-motion of the sensor body. The main idea of our work is to optimize the poses and states of two kinds of sensors with non-linear optimization methods. On the one hand, a bunch of IMU measurements are considered as a relative constraint using pre-integration and the state errors can be minimized with the help of laser pose estimation and non-linear optimizatio…
Large-Scale Tunnel Air-Ground Collaboration With FLISP: Fast LiDAR-IMU Synchronized Path Planner
Shanghai Research Institute for Intelligent Autonomous Systems, Tongji University（同济大学上海自主智能无人系统科学中心）；State Key Laboratory of Autonomous Intelligent Unmanned Systems, Tongji University（同济大学自主智能无人系统全国重点实验室）；College of Electronic and Information Engineering, Tongji University（同济大学电子与信息工程学院）
We propose an accurate and robust multi-modal sensor fusion framework, MetroLoc, towards one of the most extreme scenarios, the large-scale metro vehicle localization and mapping. MetroLoc is built atop an IMU-centric state estimator that tightly couples light detection and ranging (LiDAR), visual, and inertial information with the convenience of loosely coupled methods. The proposed framework is composed of three submodules: IMU odometry, LiDAR-inertial odometry (LIO), and Visual-inertial odometry (VIO). The IMU is treated as the primary sensor, which achieves the observations from LIO and VI…
WinTA-GIL: Windowed Trajectory Alignment for GNSS-IMU-LiDAR Heading Refinement in Intermittent Signal Environments
State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing (LIESMARS), Wuhan University（武汉大学测绘遥感信息工程国家重点实验室）；School of Remote Sensing and Information Engineering, Wuhan University（武汉大学遥感信息工程学院）；College of Engineering and Computer Science, University of Michigan-Dearborn（美国密歇根大学迪尔伯恩分校工程与计算机科学学院）
The ubiquitous planes and structural consistency are the most apparent features of indoor multi-story Buildings compared with outdoor environments. In this paper, we propose a tightly coupled LiDAR-Inertial 3D SLAM framework with plane features for the multi-story building. The framework we proposed is mainly composed of three parts: tightly coupled LiDAR-Inertial odometry, extraction of representative planes of the structure, and factor graph optimization. By building a local map and inertial measurement unit (IMU) pre-integration, we get LiDAR scan-to-local-map matching and IMU measurements,…
Radar-Inertial Odometry with Online Spatio-Temporal Calibration via Continuous-Time IMU Modeling
University of Zagreb Faculty of Electrical Engineering and Computing, Laboratory for Autonomous Systems and Mobile Robotics (LAMOR)（泽格布大学电子工程与计算学院，自主系统与移动机器人实验室）；Calirad d.o.o.（Calirad公司）
University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
We present a method of extrinsic calibration for a system of multiple inertial measurement units (IMUs) that estimates the relative pose of each IMU on a rigid body using only measurements from the IMUs themselves, without the need to prescribe the trajectory. Our method is based on solving a nonlinear least-squares problem that penalizes inconsistency between measurements from pairs of IMUs. We validate our method with experiments both in simulation and in hardware. In particular, we show that it meets or exceeds the performance -- in terms of error, success rate, and computation time -- of a…
University of Delaware（特拉华大学）；Technical University of Munich（慕尼黑工业大学）
In this paper, we study in-depth the problem of online self-calibration for robust and accurate visual-inertial state estimation. In particular, we first perform a complete observability analysis for visual-inertial navigation systems (VINS) with full calibration of sensing parameters, including IMU and camera intrinsics and IMU-camera spatial-temporal extrinsic calibration, along with readout time of rolling shutter (RS) cameras (if used). We investigate different inertial model variants containing IMU intrinsic parameters that encompass most commonly used models for low-cost inertial sensors…
Shenzhen Unity-Drive Innovation Technology Co. Ltd.（深圳一清创新科技有限公司）；Hong Kong University of Science and Technology（香港科技大学）；Hangzhou ATG Intelligent Technology Co. Ltd.（杭州ATG智能科技有限公司）
Using different sensors in an autonomous vehicle (AV) can provide multiple constraints to optimize AV location estimation. In this paper, we present a low-cost GPS-assisted LiDAR state estimation system for AVs. Firstly, we utilize LiDAR to obtain highly precise 3D geometry data. Next, we use an inertial measurement unit (IMU) to correct point cloud misalignment caused by incorrect place recognition. The estimated LiDAR odometry and IMU measurement are then jointly optimized. We use a lost-cost GPS instead of a real-time kinematic (RTK) module to refine the estimated LiDAR-inertial odometry. O…
Determining the extrinsic parameter between multiple LiDARs and cameras is essential for autonomous robots, especially for solid-state LiDARs, where each LiDAR unit has a very small Field-of-View (FoV), and multiple units are often used collectively. The majority of extrinsic calibration methods are proposed for 360$^\circ$ mechanical spinning LiDARs where the FoV overlap with other LiDAR or camera sensors is assumed. Few research works have been focused on the calibration of small FoV LiDARs and cameras nor on the improvement of the calibration speed. In this work, we consider the problem of …
Adaptive-LIO: Enhancing Robustness and Precision through Environmental Adaptation in LiDAR Inertial Odometry
State Key Laboratory of Robotics and Systems（机器人学与系统国家重点实验室）；Harbin Institute of Technology（哈尔滨工业大学）；Hangzhou Qisheng Intelligent Techology Co. Ltd.（杭州齐盛智能科技有限公司）；School of Mechatronic Engineering, China University of Mining and Technology（中国矿业大学机电工程学院）；School of Mechanical and Electrical Engineering, University of Electronic Science And Technology（电子科技大学机械与电气工程学院）；School of Electrical and Electronic Engineering, Nanyang Technological University（南洋理工大学电气与电子工程学院）
Inertial measurement unit (IMU) and odometer have been commonly-used sensors for autonomous land navigation in the global positioning system (GPS)-denied scenarios. This paper systematically proposes a versatile strategy for self-contained land vehicle navigation using the IMU and an odometer. Specifically, the paper proposes a self-calibration and refinement method for IMU/odometer integration that is able to overcome significant variation of the misalignment parameters, which are induced by many inevitable and adverse factors such as load changing, refueling and ambient temperature. An odome…
Institute of Artificial Intelligence, Huazhong University of Science and Technology（华中科技大学人工智能研究院）；Electronic Information and Communications, Huazhong University of Science and Technology（华中科技大学电子信息与通信学院）
National Institute of Advanced Industrial Science and Technology（产业技术综合研究所）
This paper presents an open source LiDAR-camera calibration toolbox that is general to LiDAR and camera projection models, requires only one pairing of LiDAR and camera data without a calibration target, and is fully automatic. For automatic initial guess estimation, we employ the SuperGlue image matching pipeline to find 2D-3D correspondences between LiDAR and camera data and estimate the LiDAR-camera transformation via RANSAC. Given the initial guess, we refine the transformation estimate with direct LiDAR-camera registration based on the normalized information distance, a mutual information…
Integrating multiple LiDAR sensors can significantly enhance a robot's perception of the environment, enabling it to capture adequate measurements for simultaneous localization and mapping (SLAM). Indeed, solid-state LiDARs can bring in high resolution at a low cost to traditional spinning LiDARs in robotic applications. However, their reduced field of view (FoV) limits performance, particularly indoors. In this paper, we propose a tightly-coupled multi-modal multi-LiDAR-inertial SLAM system for surveying and mapping tasks. By taking advantage of both solid-state and spinnings LiDARs, and buil…
