Visual Localization is an essential component in autonomous navigation. Existing approaches are either based on the visual structure from SLAM/SfM or the geometric structure from dense mapping. To take the advantages of both, in this work, we present a complete visual inertial localization system based on a hybrid map representation to reduce the computational cost and increase the positioning accuracy. Specially, we propose two modules for data association and batch optimization, respectively. To this end, we develop an efficient data association module to associate map components with local features, which takes only $2$ms to generate temporal landmarks. For batch optimization, instead of using visual factors, we develop a module to estimate a pose prior from the instant localization results to constrain poses. The experimental results on the EuRoC MAV dataset demonstrate a competitive performance compared to the state of the arts. Specially, our system achieves an average position error in 1.7 cm with 100% recall. The timings show that the proposed modules reduce the computational cost by 20-30%. We will make our implementation open source at http://github.com/hyhuang1995/gmmloc.
We present an unsupervised deep neural network approach to the fusion of RGB-D imagery with inertial measurements for absolute trajectory estimation. Our network, dubbed the Visual-Inertial-Odometry Learner (VIOLearner), learns to perform visual-inertial odometry (VIO) without inertial measurement unit (IMU) intrinsic parameters (corresponding to gyroscope and accelerometer bias or white noise) or the extrinsic calibration between an IMU and camera. The network learns to integrate IMU measurements and generate hypothesis trajectories which are then corrected online according to the Jacobians o…
This paper presents a range-aided LiDAR-inertial multi-vehicle mapping system (RaLI-Multi). Firstly, we design a multi-metric weights LiDAR-inertial odometry by fusing observations from an inertial measurement unit (IMU) and a light detection and ranging sensor (LiDAR). The degenerate level and direction are evaluated by analyzing the distribution of normal vectors of feature point clouds and are used to activate the degeneration correction module in which range measurements correct the pose estimation from the degeneration direction. We then design a multi-vehicle mapping system in which a ce…
National Laboratory of Pattern Recognition, Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所模式识别国家重点实验室）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）
Odometer-aided visual-inertial SLAM systems typically have a good performance for navigation of wheeled platforms, while they usually suffer from degenerate cases before the first turning. In this paper, firstly we perform an observability analysis w.r.t. the extrinsic parameters before the first turning, which is a complement of the existing results of observability analyses. Secondly, inspired by the above observability analyses, we propose a bidirectional trajectory computation method, by which the poses before the first turning are refined in the backward computation thread, and the real-t…
A current-aided inertial navigation framework is proposed for small autonomous underwater vehicles in long-duration operations (> 1 hour), where neither frequent surfacing nor consistent bottom-tracking are available. We instantiate this concept through mid-depth, underwater navigation. This strategy mitigates dead-reckoning uncertainty of a traditional inertial navigation system by comparing the estimate of local, ambient flow velocity with preloaded ocean current maps. The proposed navigation system is implemented through a marginalized particle filter where the vehicle's states are sequenti…
Doppler-SLAM: Doppler-Aided Radar-Inertial and LiDAR-Inertial Simultaneous Localization and Mapping
Julius-Maximilians-Universität Würzburg（乌尔姆-马克斯-普朗克大学）；Zentrum für Telematik e.V.（电信研究中心）；Nuremberg Institute of Technology Georg Simon Ohm（纽伦堡技术学院 Georg Simon Ohm）；CARIAD SE（CARIAD SE公司）；University of Bonn（波恩大学）；Center for Robotics（机器人中心）；Lamarr Institute for Machine Learning and Artificial Intelligence（Lamarr机器学习与人工智能研究所）
ONERA, Université Paris Saclay（法国航空航天实验室（ONERA），巴黎萨克雷大学）；LIRMM, Université de Montpellier, CNRS（蒙彼利埃计算机、机器人与微电子实验室，蒙彼利埃大学，法国国家科学研究中心）；DRASSM, Ministry of Culture（法国文化部水下与海底考古研究局）
This paper presents a new underwater dataset acquired from a visual-inertial-pressure acquisition system and meant to be used to benchmark visual odometry, visual SLAM and multi-sensors SLAM solutions. The dataset is publicly available and contains ground-truth trajectories for evaluation.
Centre for Automation and Robotics, Polytechnic University of Madrid（马德里理工大学自动化与机器人中心）
This article proposes a method to diminish the pose (position plus attitude) drift experienced by an SVO (Semi-Direct Visual Odometry) based visual navigation system installed onboard a UAV (Unmanned Air Vehicle) by supplementing its pose estimation non linear optimizations with priors based on the outputs of a GNSS (Global Navigation Satellite System) Denied inertial navigation system. The method is inspired in a PI (Proportional Integral) control system, in which the attitude, altitude, and rate of climb inertial outputs act as targets to ensure that the visual estimations do not deviate far…
3D Gaussian Splatting aided Localization for Large and Complex Indoor-Environments
Institute for Photogrammetry and Geoinformatics, University of Stuttgart, Germany（摄影测量与地理信息学研究所，斯图加特大学，德国）；Institute of Geomatics, University of Applied Sciences and Arts Northwestern Switzerland（地理信息研究所，北瑞士应用科学与艺术大学）
Cascaded Tightly-Coupled Observer Design for Single-Range-Aided Inertial Navigation
Department of Computer Science（计算机科学系）；Engineering, University of Quebec in Outaouais, Gatineau, QC, Canada（魁北克大学Outaouais工程学院，加蒂纳岛，QC，加拿大）；Department of Electrical and Computer Engineering（电气与计算机工程系）；Lakehead University（拉克头大学）
We present a new dataset, dedicated to the development of simultaneous localization and mapping methods for underwater vehicles navigating close to the seabed. The data sequences composing this dataset are recorded in three different environments: a harbor at a depth of a few meters, a first archaeological site at a depth of 270 meters and a second site at a depth of 380 meters. The data acquisition is performed using Remotely Operated Vehicles equipped with a monocular monochromatic camera, a low-cost inertial measurement unit, a pressure sensor and a computing unit, all embedded in a single …
In this paper, we perform a thorough observability analysis for linearized inertial navigation systems (INS) aided by exteroceptive range and/or bearing sensors (such as cameras, LiDAR and sonars) with different geometric features (points, lines and planes). While the observability of vision-aided INS (VINS) with point features has been extensively studied in the literature, we analytically show that the general aided INS with point features preserves the same observability property: that is, 4 unobservable directions, corresponding to the global yaw and the global position of the sensor platf…
Shenzhen Unity-Drive Innovation Technology Co. Ltd.（深圳一清创新科技有限公司）；Hong Kong University of Science and Technology（香港科技大学）；Hangzhou ATG Intelligent Technology Co. Ltd.（杭州ATG智能科技有限公司）
Using different sensors in an autonomous vehicle (AV) can provide multiple constraints to optimize AV location estimation. In this paper, we present a low-cost GPS-assisted LiDAR state estimation system for AVs. Firstly, we utilize LiDAR to obtain highly precise 3D geometry data. Next, we use an inertial measurement unit (IMU) to correct point cloud misalignment caused by incorrect place recognition. The estimated LiDAR odometry and IMU measurement are then jointly optimized. We use a lost-cost GPS instead of a real-time kinematic (RTK) module to refine the estimated LiDAR-inertial odometry. O…
The LiDAR and inertial sensors based localization and mapping are of great significance for Unmanned Ground Vehicle related applications. In this work, we have developed an improved LiDAR-inertial localization and mapping system for unmanned ground vehicles, which is appropriate for versatile search and rescue applications. Compared with existing LiDAR-based localization and mapping systems such as LOAM, we have two major contributions: the first is the improvement of the robustness of particle swarm filter-based LiDAR SLAM, while the second is the loop closure methods developed for global opt…
Harbin Institute of Technology Shenzhen Graduate School（哈尔滨工业大学深圳研究生院）；University of Utah（犹他大学）
A new 3D localization and mapping techinque with terrain inclination assistance is proposed in this paper to allow a robot to identify its location and build a global map in an outdoor environment. The Iterative Closest Points (ICP) algorithm and terrain inclination-based localization are combined together to achieve accurate and fast localization and mapping. Inclinations of the terrains the robot navigates are used to achieve local localization during the interval between two laser scans. Using the results of the above localization as the initial condition, the ICP algorithm is then applied …
This paper proposes an ultra-wideband (UWB) aided localization and mapping system that leverages on inertial sensor and depth camera. Inspired by the fact that visual odometry (VO) system, regardless of its accuracy in the short term, still faces challenges with accumulated errors in the long run or under unfavourable environments, the UWB ranging measurements are fused to remove the visual drift and improve the robustness. A general framework is developed which consists of three parallel threads, two of which carry out the visual-inertial odometry (VIO) and UWB localization respectively. The …
Robotics Institute, Carnegie Mellon University（卡内基梅隆大学机器人研究所）
Simultaneous Localization and Mapping (SLAM) is a fundamental task to mobile and aerial robotics. LiDAR based systems have proven to be superior compared to vision based systems due to its accuracy and robustness. In spite of its superiority, pure LiDAR based systems fail in certain degenerate cases like traveling through a tunnel. We propose Stereo Visual Inertial LiDAR (VIL) SLAM that performs better on these degenerate cases and has comparable performance on all other cases. VIL-SLAM accomplishes this by incorporating tightly-coupled stereo visual inertial odometry (VIO) with LiDAR mapping …
In this paper, we present a global navigation satellite system (GNSS) aided LiDAR-visual-inertial scheme, RailLoMer-V, for accurate and robust rail vehicle localization and mapping. RailLoMer-V is formulated atop a factor graph and consists of two subsystems: an odometer assisted LiDAR-inertial system (OLIS) and an odometer integrated Visual-inertial system (OVIS). Both the subsystem exploits the typical geometry structure on the railroads. The plane constraints from extracted rail tracks are used to complement the rotation and vertical errors in OLIS. Besides, the line features and vanishing …
University of Toronto Institute for Aerospace Studies（多伦多大学航空航天研究所）
We present a method to improve the accuracy of a zero-velocity-aided inertial navigation system (INS) by replacing the standard zero-velocity detector with a long short-term memory (LSTM) neural network. While existing threshold-based zero-velocity detectors are not robust to varying motion types, our learned model accurately detects stationary periods of the inertial measurement unit (IMU) despite changes in the motion of the user. Upon detection, zero-velocity pseudo-measurements are fused with a dead reckoning motion model in an extended Kalman filter (EKF). We demonstrate that our LSTM-bas…
Nanyang Technological University（南洋理工大学）；Australian National University（澳大利亚国立大学）
Although Simultaneous Localization and Mapping (SLAM) has been an active research topic for decades, current state-of-the-art methods still suffer from instability or inaccuracy due to feature insufficiency or its inherent estimation drift, in many civilian environments. To resolve these issues, we propose a navigation system combing the SLAM and prior-map-based localization. Specifically, we consider additional integration of line and plane features, which are ubiquitous and more structurally salient in civilian environments, into the SLAM to ensure feature sufficiency and localization robust…
We propose an accurate and robust multi-modal sensor fusion framework, MetroLoc, towards one of the most extreme scenarios, the large-scale metro vehicle localization and mapping. MetroLoc is built atop an IMU-centric state estimator that tightly couples light detection and ranging (LiDAR), visual, and inertial information with the convenience of loosely coupled methods. The proposed framework is composed of three submodules: IMU odometry, LiDAR-inertial odometry (LIO), and Visual-inertial odometry (VIO). The IMU is treated as the primary sensor, which achieves the observations from LIO and VI…
Nanyang Technological University（南洋理工大学）；Institute for Infocomm Research, A-STAR（新加坡科技研究局资讯通信研究院）
Localization can be achieved by different sensors and techniques such as a global positioning system (GPS), wifi, ultrasonic sensors, and cameras. In this paper, we focus on the laser-based localization method for unmanned aerial vehicle (UAV) applications in a GPS denied environment such as a deep tunnel system. Other than a low-cost 2D LiDAR for the planar axes, a single axis Lidar for the vertical axis as well as an inertial measurement unit (IMU) device is used to increase the reliability and accuracy of the localization performance. We present a comparative analysis of the three selected …
In this work, we propose a lightweight integrated LiDAR-Inertial SLAM system with high efficiency and a great loop closure capacity. We found that the current State-of-the-art LiDAR-Inertial SLAM system has poor performance in loop closure. The LiDAR-Inertial SLAM system often fails with the large drifting and suffers from limited efficiency when faced with large-scale circumstances. In this work, firstly, to improve the speed of the whole LiDAR-Inertial SLAM system, we have proposed a new data structure of the sparse voxel-hashing to enhance the efficiency of the LiDAR-Inertial SLAM system. S…
Department of Computer Science and Engineering, University of Quebec in Outaouais（计算机科学与工程系，魁北克大学 Outaouais分校）；Department of Electrical Engineering, Lakehead University（电气工程系，拉克希德大学）；I3S-UniCA-CNRS, University Cote d’Azur and the Insitut Universitaire de France（I3S-UniCA-CNRS，大学海岸阿兹尔分校和法国国家科学院）
In this work, we propose a novel method for performing inertial aided navigation, by using deep neural networks (DNNs). To date, most DNN inertial navigation methods focus on the task of inertial odometry, by taking gyroscope and accelerometer readings as input and regressing for integrated IMU poses (i.e., position and orientation). While this design has been successfully applied on a number of applications, it is not of theoretical performance guarantee unless patterned motion is involved. This inevitably leads to significantly reduced accuracy and robustness in certain use cases. To solve t…
In past few years we have observed an increase in the usage of RGBD sensors in mobile devices. These sensors provide a good estimate of the depth map for the camera frame, which can be used in numerous augmented reality applications. This paper presents a new visual inertial odometry (VIO) system, which uses measurements from a RGBD sensor and an inertial measurement unit (IMU) sensor for estimating the motion state of the mobile device. The resulting system is called the depth-aided VIO (DVIO) system. In this system we add the depth measurement as part of the nonlinear optimization process. S…
West Virginia University（西弗吉尼亚大学）；The Johns Hopkins University Applied Physics Laboratory（约翰斯·霍普金斯大学应用物理实验室）
In this work, we demonstrate the importance of zero velocity information for global navigation satellite system (GNSS) based navigation. The effectiveness of using the zero velocity information with zero velocity update (ZUPT) for inertial navigation applications have been shown in the literature. Here we leverage this information and add it as a position constraint in a GNSS factor graph. We also compare its performance to a GNSS/inertial navigation system (INS) coupled factor graph. We tested our ZUPT aided factor graph method on three datasets and compared it with the GNSS-only factor graph…
The Robotics Institute, Carnegie Mellon University（卡内基梅隆大学机器人研究所）
State-of-the-art forward facing monocular visual-inertial odometry algorithms are often brittle in practice, especially whilst dealing with initialisation and motion in directions that render the state unobservable. In such cases having a reliable complementary odometry algorithm enables robust and resilient flight. Using the common local planarity assumption, we present a fast, dense, and direct frame-to-frame visual-inertial odometry algorithm for downward facing cameras that minimises a joint cost function involving a homography based photometric cost and an IMU regularisation term. Via ext…
This paper proposes a unified mathematical framework for inertial measurement unit (IMU) preintegration in inertial-aided navigation system in different frames under different motion condition. The navigation state is precisely discretized as three parts: local increment, global state, and global increment. The global increment can be calculated in different frames such as local geodetic navigation frame and earth-centered-earth-fixed frame. The local increment which is referred as the IMU preintegration can be calculated under different assumptions according to the motion of the agent and the…
Institute for Aerospace Studies（航空航天研究所）；University of Toronto（多伦多大学）
The combination of ultrawideband (UWB) radios and inertial measurement units (IMU) can provide accurate positioning. To ensure reliable communication, the radios are generally mounted at the extremities of a mobile system whereas the IMUs are located closer to the center of gravity for use in control, resulting in a spatial offset between the IMU and the UWB radio. Additionally, data from heterogeneous sensors can arrive at different time instants. The systematic fusion of data from multiple sources requires the temporal offset and spatial offset between the sensors to be known. An important a…
This paper proposes the inertial localization problem, the task of estimating the absolute location from a sequence of inertial sensor measurements. This is an exciting and unexplored area of indoor localization research, where we present a rich dataset with 53 hours of inertial sensor data and the associated ground truth locations. We developed a solution, dubbed neural inertial localization (NILoc) which 1) uses a neural inertial navigation technique to turn inertial sensor history to a sequence of velocity vectors; then 2) employs a transformer-based neural architecture to find the device l…
University of Technology Sydney（悉尼科技大学）；UTS Robotics Institute（悉尼科技大学机器人研究所）
Inertial-aided systems require continuous motion excitation among other reasons to characterize the measurement biases that will enable accurate integration required for localization frameworks. This paper proposes the use of informative path planning to find the best trajectory for minimizing the uncertainty of IMU biases and an adaptive traces method to guide the planner towards trajectories which aid convergence. The key contribution is a novel regression method based on Gaussian Process (GP) to enforce continuity and differentiability between waypoints from a variant of the RRT* planning a…
Wheel-Mounted/GNSS Fusion with AI-Aided Position Updates
Autonomous Navigation and Sensor Fusion Lab（自主导航与传感器融合实验室）；Hatter Department of Marine Technologies（海洋技术系）；Charney School of Marine Sciences（海洋科学学院）；University of Haifa（海法大学）
Long-term visual localization in outdoor environment is a challenging problem, especially faced with the cross-seasonal, bi-directional tasks and changing environment. In this paper we propose a novel visual inertial localization framework that localizes against the LiDAR-built map. Based on the geometry information of the laser map, a hybrid bundle adjustment framework is proposed, which estimates the poses of the cameras with respect to the prior laser map as well as optimizes the state variables of the online visual inertial odometry system simultaneously. For more accurate cross-modal data…
Closed-loop vs. Open-loop Kalman Filter Architectures in Airborne Aided Inertial Navigation
Airbus Central Research and Technology（空客中央研究与技术中心）；Department of Engineering Cybernetics, Norwegian University of Science and Technology（挪威科技大学工程 cybernetics 系）
Western University（西安大略大学）；University of Quebec in Outaouais（魁北克大学乌塔韦校区）；Lakehead University（湖首大学）
This paper deals with the simultaneous estimation of the attitude, position and linear velocity for vision-aided inertial navigation systems. We propose a nonlinear observer on $SO(3)\times \mathbb{R}^{15}$ relying on body-frame acceleration, angular velocity and (stereo or monocular) bearing measurements of some landmarks that are constant and known in the inertial frame. Unlike the existing local Kalman-type observers, our proposed nonlinear observer guarantees almost global asymptotic stability and local exponential stability. A detailed uniform observability analysis has been conducted and…
Zhejiang University（浙江大学）；Alibaba-Zhejiang University Joint Institute of Frontier Technologies（阿里巴巴-浙江大学前沿技术联合研究院）
While visual localization or SLAM has witnessed great progress in past decades, when deploying it on a mobile robot in practice, few works have explicitly considered the kinematic (or dynamic) constraints of the real robotic system when designing state estimators. To promote the practical deployment of current state-of-the-art visual-inertial localization algorithms, in this work we propose a low-cost kinematics-constrained localization system particularly for a skid-steering mobile robot. In particular, we derive in a principle way the robot's kinematic constraints based on the instantaneous …
Indoor localization systems often fuse inertial odometry with map information via hand-defined methods to reduce odometry drift, but such methods are sensitive to noise and struggle to generalize across odometry sources. To address the robustness problem in map utilization, we propose a data-driven prior on possible user locations in a map by combining learned spatial map embeddings and temporal odometry embeddings. Our prior learns to encode which map regions are feasible locations for a user more accurately than previous hand-defined methods. This prior leads to a 49% improvement in inertial…
This article focuses on feature-based underwater localization and navigation for autonomous underwater vehicles (AUVs) using 2D imaging sonar measurements. The sparsity of underwater acoustic features and the loss of elevation angle in sonar images may introduce wrong feature matches or insufficient features for optimization-based underwater localization (i.e. under-constrained/degeneracy cases). This motivates us to propose a novel inertial-aided sliding window optimization framework to improve the estimation accuracy and the robustness to front-end outliers. Concretely, we first discriminate…
