Visual odometry (VO) is a prevalent way to deal with the relative localization problem, which is becoming increasingly mature and accurate, but it tends to be fragile under challenging environments. Comparing with classical geometry-based methods, deep learning-based methods can automatically learn effective and robust representations, such as depth, optical flow, feature, ego-motion, etc., from data without explicit computation. Nevertheless, there still lacks a thorough review of the recent advances of deep learning-based VO (Deep VO). Therefore, this paper aims to gain a deep insight on how deep learning can profit and optimize the VO systems. We first screen out a number of qualifications including accuracy, efficiency, scalability, dynamicity, practicability, and extensibility, and employ them as the criteria. Then, using the offered criteria as the uniform measurements, we detailedly evaluate and discuss how deep learning improves the performance of VO from the aspects of depth estimation, feature extraction and matching, pose estimation. We also summarize the complicated and emerging areas of Deep VO, such as mobile robots, medical robots, augmented reality and virtual reality, etc. Through the literature decomposition, analysis, and comparison, we finally put forward a number of open issues and raise some future research directions in this field.
Unsupervised Learning of Depth Estimation and Visual Odometry for Sparse Light Field Cameras
School of Aerospace, Mechanical and Mechatronic Engineering, The University of Sydney(悉尼大学航空航天、机械与机电工程学院);Sydney Institute for Robotics and Intelligent Systems(悉尼机器人与智能系统研究院)
Visual Odometry with Neuromorphic Resonator Networks
Institute of Neuroinformatics, University of Zurich and ETH Zurich(苏黎世大学与苏黎世联邦理工学院神经信息学研究所);Forschungszentrum Jülich(于利希研究中心);Accenture Labs(埃森哲实验室);Intel Neuromorphic Computing Lab, Intel Labs(英特尔实验室英特尔神经形态计算实验室);Redwood Center for Theoretical Neuroscience, UC Berkeley(加州大学伯克利分校红木理论神经科学中心);Intel Labs(英特尔实验室);ZHAW Zurich University of Applied Sciences(苏黎世应用科技大学)
KN-LIO: Geometric Kinematics and Neural Field Coupled LiDAR-Inertial Odometry
School of Electronics, Information and Electrical Engineering, Shanghai Jiao Tong University(上海交通大学电子信息与电气工程学院);School of Information and Control Engineering, China University of Mining and Technology(中国矿业大学信息与控制工程学院)
Adaptive-LIO: Enhancing Robustness and Precision through Environmental Adaptation in LiDAR Inertial Odometry
State Key Laboratory of Robotics and Systems(机器人学与系统国家重点实验室);Harbin Institute of Technology(哈尔滨工业大学);Hangzhou Qisheng Intelligent Techology Co. Ltd.(杭州齐盛智能科技有限公司);School of Mechatronic Engineering, China University of Mining and Technology(中国矿业大学机电工程学院);School of Mechanical and Electrical Engineering, University of Electronic Science And Technology(电子科技大学机械与电气工程学院);School of Electrical and Electronic Engineering, Nanyang Technological University(南洋理工大学电气与电子工程学院)
SCIPaD: Incorporating Spatial Clues into Unsupervised Pose-Depth Joint Learning
Shanghai Research Institute for Intelligent Autonomous Systems(上海智能自主系统研究院);Shanghai Institute of Intelligent Science and Technology(上海智能科学技术研究院);Frontiers Science Center for Intelligent Autonomous Systems, Tongji University(同济大学智能自主系统前沿科学中心)
Efficient and Accurate Downfacing Visual Inertial Odometry
Integrated Systems Laboratory and the Center for Project-Based Learning, ETH Zurich(集成系统实验室和项目导向学习中心,苏黎世联邦理工学院);Center for Project-Based Learning, ETH Zurich(项目导向学习中心,苏黎世联邦理工学院);Integrated Systems Laboratory, ETH Zurich(集成系统实验室,苏黎世联邦理工学院);Department of Electrical, Electronic and Information Engineering, University of Bologna(电子、电气与信息工程系,博洛尼亚大学)