We study a Visual-Inertial Navigation (VIN) problem in which a robot needs to estimate its state using an on-board camera and an inertial sensor, without any prior knowledge of the external environment. We consider the case in which the robot can allocate limited resources to VIN, due to tight computational constraints. Therefore, we answer the following question: under limited resources, what are the most relevant visual cues to maximize the performance of visual-inertial navigation? Our approach has four key ingredients. First, it is task-driven, in that the selection of the visual cues is guided by a metric quantifying the VIN performance. Second, it exploits the notion of anticipation, since it uses a simplified model for forward-simulation of robot dynamics, predicting the utility of a set of visual cues over a future time horizon. Third, it is efficient and easy to implement, since it leads to a greedy algorithm for the selection of the most relevant visual cues. Fourth, it provides formal performance guarantees: we leverage submodularity to prove that the greedy selection cannot be far from the optimal (combinatorial) selection. Simulations and real experiments on agile drones show that our approach ensures state-of-the-art VIN performance while maintaining a lean processing time. In the easy scenarios, our approach outperforms appearance-based feature selection in terms of localization errors. In the most challenging scenarios, it enables accurate visual-inertial navigation while appearance-based feature selection fails to track robot's motion during aggressive maneuvers.
Inertial navigation computation is to acquire the attitude, velocity and position information of a moving body by integrating inertial measurements from gyroscopes and accelerometers. Over half a century has witnessed great efforts in coping with the motion non-commutativity errors to accurately compute the navigation information as far as possible, so as not to compromise the quality measurements of inertial sensors. Highly dynamic applications and the forthcoming cold-atom precision inertial navigation systems demand for even more accurate inertial navigation computation. The paper gives bir…
West Virginia University（西弗吉尼亚大学）；The Johns Hopkins University Applied Physics Laboratory（约翰斯·霍普金斯大学应用物理实验室）
In this work, we demonstrate the importance of zero velocity information for global navigation satellite system (GNSS) based navigation. The effectiveness of using the zero velocity information with zero velocity update (ZUPT) for inertial navigation applications have been shown in the literature. Here we leverage this information and add it as a position constraint in a GNSS factor graph. We also compare its performance to a GNSS/inertial navigation system (INS) coupled factor graph. We tested our ZUPT aided factor graph method on three datasets and compared it with the GNSS-only factor graph…
As inertial and visual sensors are becoming ubiquitous, visual-inertial navigation systems (VINS) have prevailed in a wide range of applications from mobile augmented reality to aerial navigation to autonomous driving, in part because of the complementary sensing capabilities and the decreasing costs and size of the sensors. In this paper, we survey thoroughly the research efforts taken in this field and strive to provide a concise but complete review of the related work -- which is unfortunately missing in the literature while being greatly demanded by researchers and engineers -- in the hope…
GNSS Research Center, Wuhan University（武汉大学卫星导航定位技术研究中心）
This paper proposes a equivariant filtering (EqF) framework for the inertial-integrated state estimation problem. As the kinematic system of the inertial-integrated navigation can be naturally modeling on the matrix Lie group $SE_2(3)$, the symmetry of the Lie group can be exploited to design a equivariant filter which extends the invariant extended Kalman filtering on the group affine system. Furthermore, details of the analytic state transition matrices for left invariant error and right invariant error are given.
Shanghai Jiao Tong University（上海交通大学）；Flight Automatic Control Research Institute（飞行自动控制研究所）
Inertial navigation and attitude initialization in polar areas become a hot topic in recent years in the navigation community, as the widely-used navigation mechanization of the local level frame encounters the inherent singularity when the latitude approaches 90 degrees. Great endeavors have been devoted to devising novel navigation mechanizations such as the grid or transversal frames. This paper highlights the fact that the common Earth-frame mechanization is sufficiently good to well handle the singularity problem in polar areas. Simulation results are reported to demonstrate the singulari…
GNSS Research Center, Wuhan University（武汉大学GNSS研究中心）
This paper proposes an $SE_2(3)$ based extended Kalman filtering (EKF) framework for the inertial-integrated state estimation problem. The error representation using the straight difference of two vectors in the inertial navigation system may not be reasonable as it does not take the direction difference into consideration. Therefore, we choose to use the $SE_2(3)$ matrix Lie group to represent the state of the inertial-integrated navigation system which consequently leads to the common frame error representation. With the new velocity and position error definition, we leverage the group affin…
This paper proposes a unified mathematical framework for inertial measurement unit (IMU) preintegration in inertial-aided navigation system in different frames under different motion condition. The navigation state is precisely discretized as three parts: local increment, global state, and global increment. The global increment can be calculated in different frames such as local geodetic navigation frame and earth-centered-earth-fixed frame. The local increment which is referred as the IMU preintegration can be calculated under different assumptions according to the motion of the agent and the…
Centre for Automation and Robotics, Polytechnic University of Madrid（马德里理工大学自动化与机器人中心）
This article proposes a method to diminish the pose (position plus attitude) drift experienced by an SVO (Semi-Direct Visual Odometry) based visual navigation system installed onboard a UAV (Unmanned Air Vehicle) by supplementing its pose estimation non linear optimizations with priors based on the outputs of a GNSS (Global Navigation Satellite System) Denied inertial navigation system. The method is inspired in a PI (Proportional Integral) control system, in which the attitude, altitude, and rate of climb inertial outputs act as targets to ensure that the visual estimations do not deviate far…
The task of strapdown inertial navigation system (SINS) initial alignment is to calculate the attitude transformation matrix from body frame to navigation frame. In this paper, such attitude transformation matrix is divided into two parts through introducing the initial inertially fixed navigation frame as inertial frame. The attitude changes of the navigation frame corresponding to the defined inertial frame can be exactly calculated with known velocity and position provided by GNSS. The attitude from body frame to the defined inertial frame is estimated based on the SINS mechanization in ine…
Inertial navigation applications are usually referenced to a rotating frame. Consideration of the navigation reference frame rotation in the inertial navigation algorithm design is an important but so far less seriously treated issue, especially for ultra-high-speed flying aircraft or the future ultra-precision navigation system of several meters per hour. This paper proposes a rigorous approach to tackle the issue of navigation frame rotation in velocity/position computation by use of the newly-devised velocity/position integration formulae in the Part I companion paper. The two integration f…
Polytechnic University of Madrid（马德里理工大学）；Boeing Research & Technology Europe（波音欧洲研究与技术公司）；National University of Distance Education（国立远程教育大学）；Center for Automation and Robotics (UPM-CSIC)（自动化与机器人中心（UPM-CSIC））
This article proposes an inertial navigation algorithm intended to lower the negative consequences of the absence of GNSS (Global Navigation Satellite System) signals on the navigation of autonomous fixed wing low SWaP (Size, Weight, and Power) UAVs (Unmanned Air Vehicles). In addition to accelerometers and gyroscopes, the filter takes advantage of sensors usually present onboard these platforms, such as magnetometers, Pitot tube, and air vanes, and aims to minimize the attitude error and reduce the position drift (both horizontal and vertical) with the dual objective of improving the aircraft…
RMIT University（皇家墨尔本理工大学）；Australia National University（澳大利亚国立大学）；University of Melbourne（墨尔本大学）
Robust aiding of inertial navigation systems in GNSS-denied environments is critical for the removal of accumulated navigation error caused by the drift and bias inherent in inertial sensors. One way to perform such an aiding uses matching of geophysical measurements, such as gravimetry, gravity gradiometry or magnetometry, with a known geo-referenced map. Although simple in concept, this map matching procedure is challenging: the measurements themselves are noisy; their associated spatial location is uncertain; and the measurements may match multiple points within the map (i.e. non-unique sol…
A current-aided inertial navigation framework is proposed for small autonomous underwater vehicles in long-duration operations (> 1 hour), where neither frequent surfacing nor consistent bottom-tracking are available. We instantiate this concept through mid-depth, underwater navigation. This strategy mitigates dead-reckoning uncertainty of a traditional inertial navigation system by comparing the estimate of local, ambient flow velocity with preloaded ocean current maps. The proposed navigation system is implemented through a marginalized particle filter where the vehicle's states are sequenti…
In this letter, we present a robust, real-time, inertial navigation system (INS)-Centric GNSS-Visual-Inertial navigation system (IC-GVINS) for wheeled robot, in which the precise INS is fully utilized in both the state estimation and visual process. To improve the system robustness, the INS information is employed during the whole keyframe-based visual process, with strict outlier-culling strategy. GNSS is adopted to perform an accurate and convenient initialization of the IC-GVINS, and is further employed to achieve absolute positioning in large-scale environments. The IMU, visual, and GNSS m…
MINES ParisTech（巴黎高等矿业学院）；PSL Research University（巴黎文理研究大学）；Safran Tech（赛峰科技）
This paper proposes a real-time approach for long-term inertial navigation based only on an Inertial Measurement Unit (IMU) for self-localizing wheeled robots. The approach builds upon two components: 1) a robust detector that uses recurrent deep neural networks to dynamically detect a variety of situations of interest, such as zero velocity or no lateral slip; and 2) a state-of-the-art Kalman filter which incorporates this knowledge as pseudo-measurements for localization. Evaluations on a publicly available car dataset demonstrates that the proposed scheme may achieve a final precision of 20…
Robotics Institute（机器人研究所）；University of Technology Sydney（悉尼科技大学）
Underwater visual-inertial navigation is challenging due to the poor visibility and presence of outliers in underwater environments. The navigation performance is closely related to outlier detection and elimination. Existing methods assume the inertial odometry is accurate enough for outlier detection, which is not valid for low-cost inertial applications. We propose a novel iterative smoothing and outlier detection method aiming for underwater navigation. Using the dataset collected from an underwater robot and fiducial markers, experimental results confirm that the method can successfully e…
Inertial measurement unit (IMU) and odometer have been commonly-used sensors for autonomous land navigation in the global positioning system (GPS)-denied scenarios. This paper systematically proposes a versatile strategy for self-contained land vehicle navigation using the IMU and an odometer. Specifically, the paper proposes a self-calibration and refinement method for IMU/odometer integration that is able to overcome significant variation of the misalignment parameters, which are induced by many inevitable and adverse factors such as load changing, refueling and ambient temperature. An odome…
Norwegian University of Science and Technology（挪威科技大学）
In this paper, a computational resources-aware parameter adaptation method for visual-inertial navigation systems is proposed with the goal of enabling the improved deployment of such algorithms on computationally constrained systems. Such a capacity can prove critical when employed on ultra-lightweight systems or alongside mission critical computationally expensive processes. To achieve this objective, the algorithm proposes selected changes in the vision front-end and optimization back-end of visual-inertial odometry algorithms, both prior to execution and in real-time based on an online pro…
The Hong Kong University of Science and Technology（香港科技大学）
We present LINS, a lightweight lidar-inertial state estimator, for real-time ego-motion estimation. The proposed method enables robust and efficient navigation for ground vehicles in challenging environments, such as feature-less scenes, via fusing a 6-axis IMU and a 3D lidar in a tightly-coupled scheme. An iterated error-state Kalman filter (ESKF) is designed to correct the estimated state recursively by generating new feature correspondences in each iteration, and to keep the system computationally tractable. Moreover, we use a robocentric formulation that represents the state in a moving lo…
Long Distance GNSS-Denied Visual Inertial Navigation for Autonomous Fixed Wing Unmanned Air Vehicles: SO(3) Manifold Filter based on Virtual Vision Sensor
In this paper, the optimization-based alignment (OBA) methods are investigated with main focus on the vector observations construction procedures for the strapdown inertial navigation system (SINS). The contributions of this study are twofold. First the OBA method is extended to be able to estimate the gyroscopes biases coupled with the attitude based on the construction process of the existing OBA methods. This extension transforms the initial alignment into an attitude estimation problem which can be solved using the nonlinear filtering algorithms. The second contribution is the comprehensiv…
Washington University in St. Louis（圣路易斯华盛顿大学）；Simon Fraser University（西蒙弗雷泽大学）
This paper sets a new foundation for data-driven inertial navigation research, where the task is the estimation of positions and orientations of a moving subject from a sequence of IMU sensor measurements. More concretely, the paper presents 1) a new benchmark containing more than 40 hours of IMU sensor data from 100 human subjects with ground-truth 3D trajectories under natural human motions; 2) novel neural inertial navigation architectures, making significant improvements for challenging motion cases; and 3) qualitative and quantitative evaluations of the competing methods over three inerti…
Autonomous underwater vehicles (AUV) are commonly used in many underwater applications. Recently, the usage of multi-rotor unmanned autonomous vehicles (UAV) for marine applications is receiving more attention in the literature. Usually, both platforms employ an inertial navigation system (INS), and aiding sensors for an accurate navigation solution. In AUV navigation, Doppler velocity log (DVL) is mainly used to aid the INS, while for UAVs, it is common to use global navigation satellite systems (GNSS) receivers. The fusion between the aiding sensor and the INS requires a definition of step s…
We present a comprehensive framework for fusing measurements from multiple and generally placed accelerometers and gyroscopes to perform inertial navigation. Using the angular acceleration provided by the accelerometer array, we show that the numerical integration of the orientation can be done with second-order accuracy, which is more accurate compared to the traditional first-order accuracy that can be achieved when only using the gyroscopes. Since orientation errors are the most significant error source in inertial navigation, improving the orientation estimation reduces the overall navigat…
We present an approach to enhance wheeled planetary rover dead-reckoning localization performance by leveraging the use of zero-type constraint equations in the navigation filter. Without external aiding, inertial navigation solutions inherently exhibit cubic error growth. Furthermore, for planetary rovers that are traversing diverse types of terrain, wheel odometry is often unreliable for use in localization, due to wheel slippage. For current Mars rovers, computer vision-based approaches are generally used whenever there is a high possibility of positioning error; however, these strategies r…
In this paper we propose a new analytical preintegration theory for graph-based sensor fusion with an inertial measurement unit (IMU) and a camera (or other aiding sensors).Rather than using discrete sampling of the measurement dynamics as in current methods,we derive the closed-form solutions to the preintegration equations, yielding improved accuracy in state estimation.We advocate two new different inertial models for preintegration: (i) the model that assumes piecewise constant measurements, and (ii) the model that assumes piecewise constant local true acceleration.We show through extensiv…
Modern inertial measurements units (IMUs) are small, cheap, energy efficient, and widely employed in smart devices and mobile robots. Exploiting inertial data for accurate and reliable pedestrian navigation supports is a key component for emerging Internet-of-Things applications and services. Recently, there has been a growing interest in applying deep neural networks (DNNs) to motion sensing and location estimation. However, the lack of sufficient labelled data for training and evaluating architecture benchmarks has limited the adoption of DNNs in IMU-based tasks. In this paper, we present an…
In this paper, we propose a new strategy for learning inertial robotic navigation models. The proposed strategy enhances the generalisability of end-to-end inertial modelling, and is aimed at wheeled robotic deployments. Concretely, the paper describes the following. (1) Using precision robotics, we empirically characterise the effect of changing the sensor position during navigation on the distribution of raw inertial signals, as well as the corresponding impact on learnt latent spaces. (2) We propose neural architectures and algorithms to assimilate knowledge from an indexed set of sensor po…
This paper presents Navion, an energy-efficient accelerator for visual-inertial odometry (VIO) that enables autonomous navigation of miniaturized robots (e.g., nano drones), and virtual/augmented reality on portable devices. The chip uses inertial measurements and mono/stereo images to estimate the drone's trajectory and a 3D map of the environment. This estimate is obtained by running a state-of-the-art VIO algorithm based on non-linear factor graph optimization, which requires large irregularly structured memories and heterogeneous computation flow. To reduce the energy consumption and footp…
Baidu USA（百度美国）；University of Maryland, College Park（马里兰大学帕克分校）
We present a visual and inertial-based terrain classification network (VINet) for robotic navigation over different traversable surfaces. We use a novel navigation-based labeling scheme for terrain classification and generalization on unknown surfaces. Our proposed perception method and adaptive scheduling control framework can make predictions according to terrain navigation properties and lead to better performance on both terrain classification and navigation control on known and unknown surfaces. Our VINet can achieve 98.37% in terms of accuracy under supervised setting on known terrains a…
Galilean State Estimation for Inertial Navigation Systems with Unknown Time Delay
Control of Networked Systems Group（网络化系统控制组）；University of Klagenfurt（克莱根furt大学）；Systems Theory and Robotics Group（系统理论与机器人组）；Australian National University（澳大利亚国立大学）；I3S, CNRS, Université Côte d’Azur and Institut Universitaire de France（I3S、CNRS、坎特伯雷大学及法国大学研究院）
We propose an indoor navigation algorithm based on pedestrian dead reckoning (PDR) using an inertial measurement unit in a smartphone and map matching. The proposed indoor navigation system is user-friendly and convenient because it requires no additional device except a smartphone and works with a pedestrian in a casual posture who is walking with a smartphone in their hand. Because the performance of the PDR decreases over time, we greatly reduced the position error of the trajectory estimated by PDR using a map matching method with a known indoor map. To verify the proposed indoor navigatio…
Iterated Invariant EKF for 3D Landmark-Aided Inertial Navigation
Pontifical Catholic University of Rio de Janeiro, Rio de Janeiro, Brazil（里约热内卢天主教大学）；Dynamic Legged Systems Lab, Istituto Italiano di Tecnologia, Genova, Italy（动态腿部系统实验室，意大利理工学院）
MoRPI-PINN: A Physics-Informed Framework for Mobile Robot Pure Inertial Navigation
Autonomous Navigation and Sensor Fusion Lab (ANSFL)（自主导航与传感器融合实验室）；Hatter Department of Marine Technologies（海洋技术系）；Charney School of Marine Sciences（海洋科学学院）；University of Haifa（海法大学）
University of Central Florida（中佛罗里达大学）；Unknot.id Inc.（Unknot.id 公司）；U.S. Army CCDC SC（美国陆军作战能力发展司令部士兵中心）
Recently, data-driven inertial navigation approaches have demonstrated their capability of using well-trained neural networks to obtain accurate position estimates from inertial measurement units (IMU) measurements. In this paper, we propose a novel robust Contextual Transformer-based network for Inertial Navigation~(CTIN) to accurately predict velocity and trajectory. To this end, we first design a ResNet-based encoder enhanced by local and global multi-head self-attention to capture spatial contextual information from IMU measurements. Then we fuse these spatial representations with temporal…
University of Delaware（特拉华大学）；Technical University of Munich（慕尼黑工业大学）
In this paper, we study in-depth the problem of online self-calibration for robust and accurate visual-inertial state estimation. In particular, we first perform a complete observability analysis for visual-inertial navigation systems (VINS) with full calibration of sensing parameters, including IMU and camera intrinsics and IMU-camera spatial-temporal extrinsic calibration, along with readout time of rolling shutter (RS) cameras (if used). We investigate different inertial model variants containing IMU intrinsic parameters that encompass most commonly used models for low-cost inertial sensors…
