In this paper, we present a novel stereo visual inertial pose estimation method. Compared to the widely used filter-based or optimization-based approaches, the pose estimation process is modeled as a control system. Designed feedback or feedforward loops are introduced to achieve the stable control of the system, which include a gradient decreased feedback loop, a roll-pitch feed forward loop and a bias estimation feedback loop. This system, named FLVIS (Feedforward-feedback Loop-based Visual Inertial System), is evaluated on the popular EuRoc MAV dataset. FLVIS achieves high accuracy and robustness with respect to other state-of-the-art visual SLAM approaches. The system has also been implemented and tested on a UAV platform. The source code of this research is public to the research community.
Visibility underwater is challenging, and degrades as the distance between the subject and camera increases, making vision tasks in the forward-looking direction more difficult. We have collected underwater forward-looking stereo-vision and visual-inertial image sets in the Mediterranean and Red Sea. To our knowledge there are no other public datasets in the underwater environment acquired with this camera-sensor orientation published with ground-truth. These datasets are critical for the development of several underwater applications, including obstacle avoidance, visual odometry, 3D tracking…
Robotics Institute, Carnegie Mellon University（卡内基梅隆大学机器人研究所）
Simultaneous Localization and Mapping (SLAM) is a fundamental task to mobile and aerial robotics. LiDAR based systems have proven to be superior compared to vision based systems due to its accuracy and robustness. In spite of its superiority, pure LiDAR based systems fail in certain degenerate cases like traveling through a tunnel. We propose Stereo Visual Inertial LiDAR (VIL) SLAM that performs better on these degenerate cases and has comparable performance on all other cases. VIL-SLAM accomplishes this by incorporating tightly-coupled stereo visual inertial odometry (VIO) with LiDAR mapping …
Visual-inertial SLAM is essential for robot navigation in GPS-denied environments, e.g. indoor, underground. Conventionally, the performance of visual-inertial SLAM is evaluated with open-loop analysis, with a focus on the drift level of SLAM systems. In this paper, we raise the question on the importance of visual estimation latency in closed-loop navigation tasks, such as accurate trajectory tracking. To understand the impact of both drift and latency on visual-inertial SLAM systems, a closed-loop benchmarking simulation is conducted, where a robot is commanded to follow a desired trajectory…
This paper introduces the united monocular-stereo features into a visual-inertial tightly coupled odometry (UMS-VINS) for robust pose estimation. UMS-VINS requires two cameras and a low-cost inertial measurement unit (IMU). The UMS-VINS is an evolution of VINS-FUSION, which modifies the VINS-FUSION from the following three perspectives. 1) UMS-VINS extracts and tracks features from the sub-pixel plane to achieve better positions of the features. 2) UMS-VINS introduces additional 2-dimensional features from the left and/or right cameras. 3) If the visual initialization fails, the IMU propagatio…
CIFASIS, French Argentine International Center for Information and Systems Sciences (CONICET-UNR)（法阿国际信息与系统科学中心（国家科学技术研究委员会-罗萨里奥国立大学））；University of Zaragoza（萨拉戈萨大学）
Peking University（北京大学）；Technical University of Munich（慕尼黑工业大学）；Google Inc.（谷歌公司）
This work proposes a novel SLAM framework for stereo and visual inertial odometry estimation. It builds an efficient and robust parametrization of co-planar points and lines which leverages specific geometric constraints to improve camera pose optimization in terms of both efficiency and accuracy. %reduce the size of the Hessian matrix in the optimization. The pipeline consists of extracting 2D points and lines, predicting planar regions and filtering the outliers via RANSAC. Our parametrization scheme then represents co-planar points and lines as their 2D image coordinates and parameters of p…
Advancements in Translation Accuracy for Stereo Visual-Inertial Initialization
University of Southern California（南加州大学）；Columbia University（哥伦比亚大学）；New York University（纽约大学）；Nanyang Technological University（南洋理工大学）；Harbin Institute of Technology（哈尔滨工业大学）
ESVO2: Direct Visual-Inertial Odometry with Stereo Event Cameras
School of Robotics, Hunan University（湖南大学机器人学院）；Neuromorphic Automation and Intelligence Lab (NAIL)（神经形态自动化与智能实验室）；Department of Electronic and Computer Engineering（电子与计算机工程系）；Hong Kong University of Science and Technology（香港科学与技术大学）；TU Berlin（柏林工业大学）；Science of Intelligence Excellence Cluster（智能科学卓越集群）；Robotics Institute Germany（德国机器人研究所）；Einstein Center Digital Future（爱因斯坦中心数字未来）
German Aerospace Center (DLR)（德国航空航天中心（DLR））；Technical University of Munich（慕尼黑工业大学）
We present the DLR Planetary Stereo, Solid-State LiDAR, Inertial (S3LI) dataset, recorded on Mt. Etna, Sicily, an environment analogous to the Moon and Mars, using a hand-held sensor suite with attributes suitable for implementation on a space-like mobile rover. The environment is characterized by challenging conditions regarding both the visual and structural appearance: severe visual aliasing poses significant limitations to the ability of visual SLAM systems to perform place recognition, while the absence of outstanding structural details, joined with the limited Field-of-View of the utiliz…
GeVI-SLAM: Gravity-Enhanced Stereo Visua Inertial SLAM for Underwater Robots
School of Data Science, Chinese University of Hong Kong, Shenzhen, China（数据科学学院，香港中文大学（深圳））；School of Life Sciences, Chinese University of Hong Kong, China（生命科学学院，香港中文大学）
In this paper, we present the PerceptIn Robotics Vision System (PIRVS) system, a visual-inertial computing hardware with embedded simultaneous localization and mapping (SLAM) algorithm. The PIRVS hardware is equipped with a multi-core processor, a global-shutter stereo camera, and an IMU with precise hardware synchronization. The PIRVS software features a novel and flexible sensor fusion approach to not only tightly integrate visual measurements with inertial measurements and also to loosely couple with additional sensor modalities. It runs in real-time on both PC and the PIRVS hardware. We pe…
This paper presents Navion, an energy-efficient accelerator for visual-inertial odometry (VIO) that enables autonomous navigation of miniaturized robots (e.g., nano drones), and virtual/augmented reality on portable devices. The chip uses inertial measurements and mono/stereo images to estimate the drone's trajectory and a 3D map of the environment. This estimate is obtained by running a state-of-the-art VIO algorithm based on non-linear factor graph optimization, which requires large irregularly structured memories and heterogeneous computation flow. To reduce the energy consumption and footp…
Underwater Visual-Inertial-Acoustic-Depth SLAM with DVL Preintegration for Degraded Environments
School of Ocean Engineering and Technology & Southern Marine science and Engineering Guangdong Laboratory (Zhuhai), Sun Yat-sen University（海洋工程学院及南方海洋科学与工程广东实验室（珠海），中山大学）；Guangdong Provincial Key Laboratory of Information Technology for Deep Water Acoustics（广东省信息技术创新深水声学重点实验室）；Key Laboratory of Comprehensive Observation of Polar Environment (Sun Yat-sen University), Ministry of Education（极地环境综合观测重点实验室（中山大学），教育部）；School of Marine Science and Technology Domain, Beijing Institute of Technology (Zhuhai)（海洋科学与技术学院，北京理工大学（珠海））
The development process of high-fidelity SLAM systems depends on their validation upon reliable datasets. Towards this goal, we propose IBISCape, a simulated benchmark that includes data synchronization and acquisition APIs for telemetry from heterogeneous sensors: stereo-RGB/DVS, Depth, IMU, and GPS, along with the ground truth scene segmentation and vehicle ego-motion. Our benchmark is built upon the CARLA simulator, whose back-end is the Unreal Engine rendering a high dynamic scenery simulating the real world. Moreover, we offer 34 multi-modal datasets suitable for autonomous vehicles navig…
This paper presents ORB-SLAM3, the first system able to perform visual, visual-inertial and multi-map SLAM with monocular, stereo and RGB-D cameras, using pin-hole and fisheye lens models. The first main novelty is a feature-based tightly-integrated visual-inertial SLAM system that fully relies on Maximum-a-Posteriori (MAP) estimation, even during the IMU initialization phase. The result is a system that operates robustly in real-time, in small and large, indoor and outdoor environments, and is 2 to 5 times more accurate than previous approaches. The second main novelty is a multiple map syste…
Technical University of Munich（慕尼黑工业大学）；Imperial College London（伦敦帝国学院）
Robotic applications are continuously striving towards higher levels of autonomy. To achieve that goal, a highly robust and accurate state estimation is indispensable. Combining visual and inertial sensor modalities has proven to yield accurate and locally consistent results in short-term applications. Unfortunately, visual-inertial state estimators suffer from the accumulation of drift for long-term trajectories. To eliminate this drift, global measurements can be fused into the state estimation pipeline. The most known and widely available source of global measurements is the Global Position…
Sensor Configuration Matters: A Systematic Evaluation of Multimodal SLAM on Quadruped Robots
Machine Intelligence and Robotics Lab, Karlsruhe Institute of Technology (KIT)（卡尔斯鲁厄理工学院智能机器人实验室）；Institute for Intelligent Systems, Esslingen University of Applied Sciences（埃森堡应用科学大学智能系统研究所）；Department of Computer Science, University of Freiburg（弗赖堡大学计算机科学系）
DIVO: Continuous-time DVL-Inertial-Visual Odometry for Unmanned Underwater Vehicles
Voyis Imaging Inc.（Voyis影像公司）；Natural Sciences and Engineering Research Council of Canada (NSERC)（加拿大自然科学与工程研究理事会（NSERC））；McGill University（麦吉尔大学）；McGill Engineering Doctoral Award (MEDA) Program（麦吉尔大学工程博士奖（MEDA）项目）
Oxford Robotics Institute（牛津机器人研究所）；University of Oxford（牛津大学）
In this paper we present a large dataset with a variety of mobile mapping sensors collected using a handheld device carried at typical walking speeds for nearly 2.2 km through New College, Oxford. The dataset includes data from two commercially available devices - a stereoscopic-inertial camera and a multi-beam 3D LiDAR, which also provides inertial measurements. Additionally, we used a tripod-mounted survey grade LiDAR scanner to capture a detailed millimeter-accurate 3D map of the test location (containing $\sim$290 million points). Using the map we inferred centimeter-accurate 6 Degree of F…
ETH Zurich（苏黎世联邦理工学院）；Voliro Airborne Robotics（Voliro 空中机器人公司）；Sevensense Robotics（Sevensense 机器人公司）
Robust and accurate pose estimation is crucial for many applications in mobile robotics. Extending visual Simultaneous Localization and Mapping (SLAM) with other modalities such as an inertial measurement unit (IMU) can boost robustness and accuracy. However, for a tight sensor fusion, accurate time synchronization of the sensors is often crucial. Changing exposure times, internal sensor filtering, multiple clock sources and unpredictable delays from operation system scheduling and data transfer can make sensor synchronization challenging. In this paper, we present VersaVIS, an Open Versatile …
RUSSO: Robust Underwater SLAM with Sonar Optimization against Visual Degradation
School of Mechanical Engineering and Automation, Harbin Institute of Technology, Shenzhen（哈尔滨工业大学（深圳）机械工程与自动化学院）；School of Electrical and Information Engineering, Tianjin University（天津大学电气自动化与信息工程学院）
AERO-VIS: Asynchronous Event-based Real-time Onboard Visual-Inertial SLAM
Mobile Robotics Lab, ETH Zürich（苏黎世联邦理工学院移动机器人实验室）；Technical University of Munich（慕尼黑技术大学）；Munich Institute of Robotics and Machine Intelligence (MIRMI)（慕尼黑机器人与机器智能研究所）
A Consistency-Improved LiDAR-Inertial Bundle Adjustment
Aerospace Information Research Institute, Chinese Academy of Sciences（中国科学院航空信息研究所）；University of Chinese Academy of Sciences（中国科学院大学）；Aerospace Information Technology University（航空信息科技大学）；QiLu Aerospace Information Research Institute, Chinese Academy of Sciences（齐鲁航空信息研究所）；School of Electronic, Electrical and Communication Engineering（电子电气与通信工程学院）
本文提出改进一致性的 LiDAR-惯性束调整方法，通过定制参数化和估计器提升 SLAM 系统的准确性和可观测性。
GRASP Lab, University of Pennsylvania（宾夕法尼亚大学GRASP实验室）
In recent years, vision-aided inertial odometry for state estimation has matured significantly. However, we still encounter challenges in terms of improving the computational efficiency and robustness of the underlying algorithms for applications in autonomous flight with micro aerial vehicles in which it is difficult to use high quality sensors and pow- erful processors because of constraints on size and weight. In this paper, we present a filter-based stereo visual inertial odometry that uses the Multi-State Constraint Kalman Filter (MSCKF) [1]. Previous work on stereo visual inertial odomet…
University of Maryland Institute for Advanced Computer Studies（马里兰大学高级计算机研究所）；University of Maryland, College Park（马里兰大学帕克分校）
A new event camera dataset, EVIMO2, is introduced that improves on the popular EVIMO dataset by providing more data, from better cameras, in more complex scenarios. As with its predecessor, EVIMO2 provides labels in the form of per-pixel ground truth depth and segmentation as well as camera and object poses. All sequences use data from physical cameras and many sequences feature multiple independently moving objects. Typically, such labeled data is unavailable in physical event camera datasets. Thus, EVIMO2 will serve as a challenging benchmark for existing algorithms and rich training set for…
Autonomous Systems Lab, ETH Zurich（苏黎世联邦理工学院自主系统实验室）
This paper proposes a computationally efficient method to estimate the time-varying relative pose between two visual-inertial sensor rigs mounted on the flexible wings of a fixed-wing unmanned aerial vehicle (UAV). The estimated relative poses are used to generate highly accurate depth maps in real-time and can be employed for obstacle avoidance in low-altitude flights or landing maneuvers. The approach is structured as follows: Initially, a wing model is identified by fitting a probability density function to measured deviations from the nominal relative baseline transformation. At run-time, …
M-SEVIQ: A Multi-band Stereo Event Visual-Inertial Quadruped-based Dataset for Perception under Rapid Motion and Challenging Illumination
Shanghai Key Laboratory of Navigation & Location Based Services, Shanghai Jiao Tong University（上海导航与基于位置的服务关键实验室，上海交通大学）；Beijing Aerospace Microsystem and Information Technology Research Institute（北京航天微系统与信息技术研究所）
Institute for Intelligent Systems, Esslingen University of Applied Sciences（智能系统研究所，埃斯林根应用科学大学）；Department of Computer Science, University of Freiburg（计算机科学系，弗赖堡大学）；ANDREAS STIHL AG & Co. KG（安德烈亚斯·施塔尔有限公司）
MINES ParisTech, PSL Research University（巴黎高等矿业学院，巴黎文理研究大学）；Safran Tech, Groupe Safran（赛峰集团赛峰科技）
In this paper we propose a novel accurate method for dead-reckoning of wheeled vehicles based only on an Inertial Measurement Unit (IMU). In the context of intelligent vehicles, robust and accurate dead-reckoning based on the IMU may prove useful to correlate feeds from imaging sensors, to safely navigate through obstructions, or for safe emergency stops in the extreme case of exteroceptive sensors failure. The key components of the method are the Kalman filter and the use of deep neural networks to dynamically adapt the noise parameters of the filter. The method is tested on the KITTI odometr…
Control-Barrier-Aided Teleoperation with Visual-Inertial SLAM for Safe MAV Navigation in Complex Environments
Learning Systems and Robotics Lab, School of Computation, Information and Technology, Technical University of Munich（学习系统与机器人实验室，计算、信息与技术学院，慕尼黑技术大学）；Smart Robotics Lab, School of Computation, Information and Technology, Technical University of Munich（智能机器人实验室，计算、信息与技术学院，慕尼黑技术大学）；Smart Robotics Lab, Department of Computing, Imperial College London（智能机器人实验室，计算系，伦敦帝国理工学院）；Munich Institute of Robotics and Machine Intellig（慕尼黑机器人与机器智能研究所）
Korea Advanced Institute of Science and Technology（韩国科学技术院）；Korea Electronics Technology Institute (KETI)（韩国电子技术研究院）；KI-AI；KIR at KAIST（韩国科学技术院KIR）
This paper presents benchmark tests of various visual(-inertial) odometry algorithms on NVIDIA Jetson platforms. The compared algorithms include mono and stereo, covering Visual Odometry (VO) and Visual-Inertial Odometry (VIO): VINS-Mono, VINS-Fusion, Kimera, ALVIO, Stereo-MSCKF, ORB-SLAM2 stereo, and ROVIO. As these methods are mainly used for unmanned aerial vehicles (UAVs), they must perform well in situations where the size of the processing board and weight is limited. Jetson boards released by NVIDIA satisfy these constraints as they have a sufficiently powerful central processing unit (…
Commercial visual-inertial odometry (VIO) systems have been gaining attention as cost-effective, off-the-shelf six degrees of freedom (6-DoF) ego-motion tracking methods for estimating accurate and consistent camera pose data, in addition to their ability to operate without external localization from motion capture or global positioning systems. It is unclear from existing results, however, which commercial VIO platforms are the most stable, consistent, and accurate in terms of state estimation for indoor and outdoor robotic applications. We assess four popular proprietary VIO systems (Apple A…
