This article describes an algorithm that provides visual odometry estimates from sequential pairs of RGBD images. The key contribution of this article on RGBD odometry is that it provides both an odometry estimate and a covariance for the odometry parameters in real-time via a representative covariance matrix. Accurate, real-time parameter covariance is essential to effectively fuse odometry measurements into most navigation systems. To date, this topic has seen little treatment in research which limits the impact existing RGBD odometry approaches have for localization in these systems. Covariance estimates are obtained via a statistical perturbation approach motivated by real-world models of RGBD sensor measurement noise. Results discuss the accuracy of our RGBD odometry approach with respect to ground truth obtained from a motion capture system and characterizes the suitability of this approach for estimating the true RGBD odometry parameter uncertainty.
To measure system states and local environment directly with high precision, expensive sensors are required. However, highly accurate system states and environmental perception can also be achieved using data fusion techniques and digital maps. One crucial task of multi-sensor state estimation is to project different sensor measurements into the same temporal, spatial and physical domain, estimate their covariance matrices as well as the exclusion of erroneous measurements. This paper presents a generic approach for robust estimation of vehicle movement (odometry). We will shortly present our …
Localization is an essential technique in mobile robotics. In a complex environment, it is necessary to fuse different localization modules to obtain more robust results, in which the error model plays a paramount role. However, exteroceptive sensor-based odometries (ESOs), such as LiDAR/visual odometry, often deliver results with scene-related error, which is difficult to model accurately. To address this problem, this research designs a scene-aware error model for ESO, based on which a multimodal localization fusion framework is developed. In addition, an end-to-end learning method is propos…
Peking University（北京大学）；AVIC (Aviation Industry Corporation of China) Beijing Changcheng Aeronautical Measurement and Control Technology Research Institute（中国航空工业集团公司北京长城航空测控技术研究所）
We propose a novel deep visual odometry (VO) method that considers global information by selecting memory and refining poses. Existing learning-based methods take the VO task as a pure tracking problem via recovering camera poses from image snippets, leading to severe error accumulation. Global information is crucial for alleviating accumulated errors. However, it is challenging to effectively preserve such information for end-to-end systems. To deal with this challenge, we design an adaptive memory module, which progressively and adaptively saves the information from local to global in a neur…
The present paper reviews the classical problem of free-form curve registration and applies it to an efficient RGBD visual odometry system called Canny-VO, as it efficiently tracks all Canny edge features extracted from the images. Two replacements for the distance transformation commonly used in edge registration are proposed: Approximate Nearest Neighbour Fields and Oriented Nearest Neighbour Fields. 3D2D edge alignment benefits from these alternative formulations in terms of both efficiency and accuracy. It removes the need for the more computationally demanding paradigms of datato-model re…
The paper presents a direct visual-inertial odometry system. In particular, a tightly coupled nonlinear optimization based method is proposed by integrating the recent advances in direct dense tracking and Inertial Measurement Unit (IMU) pre-integration, and a factor graph optimization is adapted to estimate the pose of the camera and rebuild a semi-dense map. Two sliding windows are maintained in the proposed approach. The first one, based on Direct Sparse Odometry (DSO), is to estimate the depths of candidate points for mapping and dense visual tracking. In the second one, measurements from …
This paper presents CAPE, a method to extract planes and cylinder segments from organized point clouds, which processes 640x480 depth images on a single CPU core at an average of 300 Hz, by operating on a grid of planar cells. While, compared to state-of-the-art plane extraction, the latency of CAPE is more consistent and 4-10 times faster, depending on the scene, we also demonstrate empirically that applying CAPE to visual odometry can improve trajectory estimation on scenes made of cylindrical surfaces (e.g. tunnels), whereas using a plane extraction approach that is not curve-aware deterior…
In past few years we have observed an increase in the usage of RGBD sensors in mobile devices. These sensors provide a good estimate of the depth map for the camera frame, which can be used in numerous augmented reality applications. This paper presents a new visual inertial odometry (VIO) system, which uses measurements from a RGBD sensor and an inertial measurement unit (IMU) sensor for estimating the motion state of the mobile device. The resulting system is called the depth-aided VIO (DVIO) system. In this system we add the depth measurement as part of the nonlinear optimization process. S…
MVOFormer: Flow-Semantic Transformer for Robust Monocular Visual Odometry
State Key Laboratory of Fluid Power and Mechatronic Systems, Zhejiang University（浙江大学流体动力与机电系统国家重点实验室）；Zhejiang Key Laboratory of Industrial Big Data and Robot Intelligent Systems（浙江省工业大数据与机器人智能系统重点实验室）；School of Mechanical Engineering, Zhejiang University（浙江大学机械工程学院）；Robotics Institute, Zhejiang University（浙江大学机器人研究院）；School of Artificial Intelligence and Robotics, Hunan University（湖南大学人工智能与机器人学院）；Rural Health Research Institute, Charles Sturt University（查尔斯特大学农村健康研究所）；University College London（伦敦大学学院）
Singapore University of Technology and Design（新加坡科技设计大学）；Southwest University of Science and Technology（西南科技大学）
To accomplish task efficiently in a multiple robots system, a problem that has to be addressed is Simultaneous Localization and Mapping (SLAM). LiDAR (Light Detection and Ranging) has been used for many SLAM solutions due to its superb accuracy, but its performance degrades in featureless environments, like tunnels or long corridors. Centralized SLAM solves the problem with a cloud server, which requires a huge amount of computational resources and lacks robustness against central node failure. To address these issues, we present a distributed SLAM solution to estimate the trajectory of a grou…
We have proposed, to the best of our knowledge, the first-of-its-kind LiDAR-Inertial-Visual-Fused simultaneous localization and mapping (SLAM) system with a strong place recognition capacity. Our proposed SLAM system is consist of visual-inertial odometry (VIO) and LiDAR inertial odometry (LIO) subsystems. We propose the LIO subsystem utilizing the measurement from the LiDAR and the inertial sensors to build the local odometry map, and propose the VIO subsystem which takes in the visual information to construct the 2D-3D associated map. Then, we propose an iterative Kalman Filter-based optimiz…
This paper proposes an ultra-wideband (UWB) aided localization and mapping system that leverages on inertial sensor and depth camera. Inspired by the fact that visual odometry (VO) system, regardless of its accuracy in the short term, still faces challenges with accumulated errors in the long run or under unfavourable environments, the UWB ranging measurements are fused to remove the visual drift and improve the robustness. A general framework is developed which consists of three parallel threads, two of which carry out the visual-inertial odometry (VIO) and UWB localization respectively. The …
Super-LIO: A Robust and Efficient LiDAR-Inertial Odometry System with a Compact Mapping Strategy
Department of Mechanical Engineering, East China University of Science and Technology（东华大学机械工程学院）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；University of Hong Kong（香港大学）
Con-DSO: Learning Short-Horizon Consistency Priors for RGB-D Direct Sparse Odometry
School of Information Science, Japan Advanced Institute of Science and Technology（信息科学学系，日本科学技术先进研究院）；College of Information Engineering, Shenyang University of Chemical Technology（信息工程学院，沈阳化学工业大学）
Learning to Anchor Visual Odometry: KAN-Based Pose Regression for Planetary Landing
University of Chinese Academy of Sciences（中国科学院大学）；Technology and Engineering Center for Space Utilization, Chinese Academy of Sciences（中国科学院空间利用技术与工程中心）
Columbia University（哥伦比亚大学）；U.S. Army Research Laboratory（美国陆军研究实验室）
We present a robot navigation system that uses an imitation learning framework to successfully navigate in complex environments. Our framework takes a pre-built 3D scan of a real environment and trains an agent from pre-generated expert trajectories to navigate to any position given a panoramic view of the goal and the current visual input without relying on map, compass, odometry, or relative position of the target at runtime. Our end-to-end trained agent uses RGB and depth (RGBD) information and can handle large environments (up to $1031m^2$) across multiple rooms (up to $40$) and generalize…
GNSS/Multi-Sensor Fusion Using Continuous-Time Factor Graph Optimization for Robust Localization
Institute of Automatic Control, Faculty of Mechanical Engineering, RWTH Aachen University（自动控制研究所，机械工程学院，亚琛RWTH大学）；Department of BioMechanical Engineering, Delft University of Technology（生物机械工程系，代尔夫特技术大学）；University of Toronto Robotics Institute（多伦多大学机器人研究所）
Technical University of Munich（慕尼黑工业大学）；Artisense；Karlsruhe University of Applied Sciences（卡尔斯鲁厄应用科学大学）
In this paper, we present TANDEM a real-time monocular tracking and dense mapping framework. For pose estimation, TANDEM performs photometric bundle adjustment based on a sliding window of keyframes. To increase the robustness, we propose a novel tracking front-end that performs dense direct image alignment using depth maps rendered from a global model that is built incrementally from dense depth predictions. To predict the dense depth maps, we propose Cascade View-Aggregation MVSNet (CVA-MVSNet) that utilizes the entire active keyframe window by hierarchically constructing 3D cost volumes wit…
In this paper, a complete framework for Autonomous Self Driving is implemented. LIDAR, Camera and IMU sensors are used together. The entire data communication is managed using Robot Operating System which provides a robust platform for implementation of Robotics Projects. Jetson Nano is used to provide powerful on-board processing capabilities. Sensor fusion is performed on the data received from the different sensors to improve the accuracy of the decision making and inferences that we derive from the data. This data is then used to create a localized map of the environment. In this step, the…
