Loop closure based on camera images provides excellent results on benchmarking datasets, but might struggle in real-world adverse weather conditions like direct sun, rain, fog, or just darkness at night. In automotive applications, the sensory setups include 3D LiDARs that provide information complementary to cameras. The presented article focuses on the evaluation of camera-based, LiDAR-based, and joint camera-LiDAR-based loop closures applying a similar processing pipeline consisting of a neural network under varying weather conditions using the newly available USyd dataset. The experiments performed on the same trajectories in diverse weather conditions over 50 weeks prove that a 16-line 3D LiDAR can be used to supplement image-based loop closure to increase loop closure performance. This proves that there is a need for more research into loop closures performed with multi-sensory setups.
Institute of Sensors, Signals and Systems, Heriot-Watt University（赫瑞-瓦特大学传感器、信号与系统研究所）
Datasets for autonomous cars are essential for the development and benchmarking of perception systems. However, most existing datasets are captured with camera and LiDAR sensors in good weather conditions. In this paper, we present the RAdar Dataset In Adverse weaThEr (RADIATE), aiming to facilitate research on object detection, tracking and scene understanding using radar sensing for safe autonomous driving. RADIATE includes 3 hours of annotated radar images with more than 200K labelled road actors in total, on average about 4.6 instances per radar image. It covers 8 different categories of a…
In this work, we present LIBRE: LiDAR Benchmarking and Reference, a first-of-its-kind dataset featuring 10 different LiDAR sensors, covering a range of manufacturers, models, and laser configurations. Data captured independently from each sensor includes three different environments and configurations: static targets, where objects were placed at known distances and measured from a fixed position within a controlled environment; adverse weather, where static obstacles were measured from a moving vehicle, captured in a weather chamber where LiDARs were exposed to different conditions (fog, rain…
Reliable and accurate localization is crucial for mobile autonomous systems. Pole-like objects, such as traffic signs, poles, lamps, etc., are ideal landmarks for localization in urban environments due to their local distinctiveness and long-term stability. In this paper, we present a novel, accurate, and fast pole extraction approach that runs online and has little computational demands such that this information can be used for a localization system. Our method performs all computations directly on range images generated from 3D LiDAR scans, which avoids processing 3D point cloud explicitly …
OmniLiDAR: A Unified Diffusion Framework for Multi-Domain 3D LiDAR Generation
College of Computer Science and Artificial Intelligence, Fudan University（复旦大学计算机科学与人工智能学院）；School of Computing, Department of Computer Science, National University of Singapore（新加坡国立大学计算机学院）；College of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics（南京航空航天大学计算机科学与技术学院）；Technical University of Munich（慕尼黑技术大学）；Nanjing University of Science and Technology（南京理工大学）；Shanghai AI Laboratory（上海人工智能实验室）；University of Sydney（悉尼大学）；The Chinese University of Hong Kong, Hong Kong SAR（香港中文大学（深圳））
Generalizing Unsupervised Lidar Odometry Model from Normal to Snowy Weather Conditions
Shanghai Polytechnic University（上海理工大学）；Singapore Management University（新加坡国立大学）；Nanjing University of Science and Technology（南京理工大学）；University of Macau（澳门大学）
In this work, we propose a lightweight integrated LiDAR-Inertial SLAM system with high efficiency and a great loop closure capacity. We found that the current State-of-the-art LiDAR-Inertial SLAM system has poor performance in loop closure. The LiDAR-Inertial SLAM system often fails with the large drifting and suffers from limited efficiency when faced with large-scale circumstances. In this work, firstly, to improve the speed of the whole LiDAR-Inertial SLAM system, we have proposed a new data structure of the sparse voxel-hashing to enhance the efficiency of the LiDAR-Inertial SLAM system. S…
An Iterative Task-Driven Framework for Resilient LiDAR Place Recognition in Adverse Weather
School of Electronic and Information Engineering, Harbin Institute of Technology (Shenzhen)（哈尔滨工业大学（深圳）电子与信息工程学院）；College of Intelligence Science and Technology, National University of Defense Technology（国防科技大学智能科学与技术学院）；School of Information Communication Technology, Shenzhen Institute of Information Technology（深圳信息职业技术学院信息通信技术学院）；Harvard AI and Robotics Lab, Harvard University（哈佛大学AI与机器人实验室）；State Key Laboratory of Robotics and System, Harbin Institute of Technology（哈尔滨工业大学机器人系统国家重点实验室）；School of Electronic and Communication Engineering, Shenzhen Polytechnic University（深圳职业技术大学电子与通信工程学院）
LiDAR odometry can achieve accurate vehicle pose estimation for short driving range or in small-scale environments, but for long driving range or in large-scale environments, the accuracy deteriorates as a result of cumulative estimation errors. This drawback necessitates the inclusion of loop closure detection in a SLAM framework to suppress the adverse effects of cumulative errors. To improve the accuracy of pose estimation, we propose a new LiDAR-based SLAM method which uses F-LOAM as LiDAR odometry, Scan Context for loop closure detection, and GTSAM for global optimization. In our approach…
Daimler AG（戴姆勒股份公司）；Karlsruhe Institute of Technology (KIT)（卡尔斯鲁厄理工学院）
Lidar sensors are frequently used in environment perception for autonomous vehicles and mobile robotics to complement camera, radar, and ultrasonic sensors. Adverse weather conditions are significantly impacting the performance of lidar-based scene understanding by causing undesired measurement points that in turn effect missing detections and false positives. In heavy rain or dense fog, water drops could be misinterpreted as objects in front of the vehicle which brings a mobile robot to a full stop. In this paper, we present the first CNN-based approach to understand and filter out such adver…
BEV-LIO(LC): BEV Image Assisted LiDAR-Inertial Odometry with Loop Closure
School of Computer Science and Technology, Guangdong University of Technology（计算机科学与技术学院，广东技术大学）；School of Electrical and Electronic Engineering, Nanyang Technological University（电子与电气工程学院，南洋理工大学）
University of Freiburg（弗赖堡大学）；University of Technology Nuremberg（纽伦堡工业大学）
A key component of graph-based SLAM systems is the ability to detect loop closures in a trajectory to reduce the drift accumulated over time from the odometry. Most LiDAR-based methods achieve this goal by using only the geometric information, disregarding the semantics of the scene. In this work, we introduce PADLoC for joint loop closure detection and registration in LiDAR-based SLAM frameworks. We propose a novel transformer-based head for point cloud matching and registration, and to leverage panoptic information during training time. In particular, we propose a novel loss function that re…
Nanjing University of Science and Technology（南京理工大学）；University of Macau（澳门大学）；MIT（麻省理工学院）
In this paper, a global descriptor for a LiDAR point cloud, called LiDAR Iris, is proposed for fast and accurate loop-closure detection. A binary signature image can be obtained for each point cloud after several LoG-Gabor filtering and thresholding operations on the LiDAR-Iris image representation. Given two point clouds, their similarities can be calculated as the Hamming distance of two corresponding binary signature images extracted from the two point clouds, respectively. Our LiDAR-Iris method can achieve a pose-invariant loop-closure detection at a descriptor level with the Fourier trans…
NDT-Map-Code: A 3D global descriptor for real-time loop closure detection in lidar SLAM
The College of Mechanical and Vehicle Engineering, Chongqing University（重庆大学机械与车辆工程学院）；NIO（蔚来）；The State Key Lab of Mechanical Transmissions, Chongqing University（重庆大学机械传动国家重点实验室）；Department of Computer Science, The University of Sheffield（谢菲尔德大学计算机科学系）
Stable feature extraction is the key for the Loop closure detection (LCD) task in the simultaneously localization and mapping (SLAM) framework. In our paper, the feature extraction is operated by using a generative adversarial networks (GANs) based unsupervised learning. GANs are powerful generative models, however, GANs based adversarial learning suffers from training instability. We find that the data-code joint distribution in the adversarial learning is a more complex manifold than in the original GANs. And the loss function that drive the attractive force between synthesis and target dist…
Multi-Mapcher: Loop Closure Detection-Free Heterogeneous LiDAR Multi-Session SLAM Leveraging Outlier-Robust Registration for Autonomous Vehicles
Laboratory for Information & Decision Systems, Massachusetts Institute of Technology（信息与决策系统实验室，麻省理工学院）；School of Electrical Engineering, KAIST（电气工程学院，韩国科学技术院）
Robotics and Autonomous Systems Group, DATA61, CSIRO（联邦科学与工业研究组织 DATA61 机器人与自主系统组）；School of Electrical Engineering and Computer Science, Queensland University of Technology (QUT)（昆士兰科技大学电气工程与计算机科学学院）；Division of Smart Automotive Engineering, Sun Moon University（鲜文大学智能汽车工程学部）
Map-centric SLAM utilizes elasticity as a means of loop closure. This approach reduces the cost of loop closure while still provides large-scale fusion-based dense maps, when compared to the trajectory-centric SLAM approaches. In this paper, we present a novel framework for 3D LiDAR-based map-centric SLAM. Having the advantages of a map-centric approach, our method exhibits new features to overcome the shortcomings of existing systems, associated with multi-modal sensor fusion and LiDAR motion distortion. This is accomplished through the use of a local Continuous-Time (CT) trajectory represent…
This paper presents a loop closure method to correct the long-term drift in LiDAR odometry and mapping (LOAM). Our proposed method computes the 2D histogram of keyframes, a local map patch, and uses the normalized cross-correlation of the 2D histograms as the similarity metric between the current keyframe and those in the map. We show that this method is fast, invariant to rotation, and produces reliable and accurate loop detection. The proposed method is implemented with careful engineering and integrated into the LOAM algorithm, forming a complete and practical system ready to use. To benefi…
Shenyang Institute of Automation, Chinese Academy of Sciences（中国科学院沈阳自动化研究所）；University of Chinese Academy of Sciences（中国科学院大学）；University of Auckland（奥克兰大学）
Loop Closure Detection (LCD) is the essential module in the simultaneous localization and mapping (SLAM) task. In the current appearance-based SLAM methods, the visual inputs are usually affected by illumination, appearance and viewpoints changes. Comparing to the visual inputs, with the active property, light detection and ranging (LiDAR) based point-cloud inputs are invariant to the illumination and appearance changes. In this paper, we extract 3D voxel maps and 2D top view maps from LiDAR inputs, and the former could capture the local geometry into a simplified 3D voxel format, the later co…
Loop closure detection (LCD) is the key module in appearance based simultaneously localization and mapping (SLAM). However, in the real life, the appearance of visual inputs are usually affected by the illumination changes and texture changes under different weather conditions. Traditional methods in LCD usually rely on handcraft features, however, such methods are unable to capture the common descriptions under different weather conditions, such as rainy, foggy and sunny. Furthermore, traditional handcraft features could not capture the highly level understanding for the local scenes. In this…
Enhancing Lidar-based Object Detection in Adverse Weather using Offset Sequences in Time
Institute for Automotive Engineering RWTH Aachen University（亚琛工业大学汽车工程研究所）；Institute of Imaging and Computer Vision RWTH Aachen University（亚琛工业大学图像与计算机视觉研究所）
Towards Generalized Range-View LiDAR Segmentation in Adverse Weather
University of Electronic Science and Technology of China（电子科技大学）；Dalian University of Technology（大连理工大学）；Lancaster University（兰卡斯特大学）；Nanyang Technological University（南洋理工大学）；Tongji University（同济大学）
针对距离视图 LiDAR 分割在恶劣天气下泛化不足的问题，提出轻量模块化 GAS 与 RDC 双分支框架，在不改动核心架构的情况下显著提升鲁棒性且推理开销极小。
University of Bonn（波恩大学）；Fraunhofer FKIE（弗劳恩霍夫通信、信息处理与人机工程研究所）；Autonomous Intelligent Driving GmbH（自动驾驶有限公司）
Simultaneous localization and mapping (SLAM) is a fundamental capability required by most autonomous systems. In this paper, we address the problem of loop closing for SLAM based on 3D laser scans recorded by autonomous cars. Our approach utilizes a deep neural network exploiting different cues generated from LiDAR data for finding loop closures. It estimates an image overlap generalized to range images and provides a relative yaw angle estimate between pairs of scans. Based on such predictions, we tackle loop closure detection and integrate our approach into an existing SLAM system to improve…
LiDAR-based SLAM system is admittedly more accurate and stable than others, while its loop closure detection is still an open issue. With the development of 3D semantic segmentation for point cloud, semantic information can be obtained conveniently and steadily, essential for high-level intelligence and conductive to SLAM. In this paper, we present a novel semantic-aided LiDAR SLAM with loop closure based on LOAM, named SA-LOAM, which leverages semantics in odometry as well as loop closure detection. Specifically, we propose a semantic-assisted ICP, including semantically matching, downsamplin…
A Simultaneous Localization and Mapping (SLAM) system must be robust to support long-term mobile vehicle and robot applications. However, camera and LiDAR based SLAM systems can be fragile when facing challenging illumination or weather conditions which degrade their imagery and point cloud data. Radar, whose operating electromagnetic spectrum is less affected by environmental changes, is promising although its distinct sensing geometry and noise characteristics bring open challenges when being exploited for SLAM. % However, there are still open challenges since most existing visual and LiDAR …
DIDLM: A SLAM Dataset for Difficult Scenarios Featuring Infrared, Depth Cameras, LIDAR, 4D Radar, and Others under Adverse Weather, Low Light Conditions, and Rough Roads
School of Information Science and Technology, Northwest University（信息科学与技术学院，西北大学）；Department of Electrical and Computer Engineering, The University of British Columbia（电气与计算机工程系，不列颠哥伦比亚大学）
Oxford Robotics Institute, University of Oxford（牛津大学牛津机器人研究所）
In this paper, we present a factor-graph LiDAR-SLAM system which incorporates a state-of-the-art deeply learned feature-based loop closure detector to enable a legged robot to localize and map in industrial environments. These facilities can be badly lit and comprised of indistinct metallic structures, thus our system uses only LiDAR sensing and was developed to run on the quadruped robot's navigation PC. Point clouds are accumulated using an inertial-kinematic state estimator before being aligned using ICP registration. To close loops we use a loop proposal mechanism which matches individual …
University of Freiburg（弗莱堡大学）；Università degli studi di Milano - Bicocca（米兰比可卡大学）
Loop closure detection is an essential component of Simultaneous Localization and Mapping (SLAM) systems, which reduces the drift accumulated over time. Over the years, several deep learning approaches have been proposed to address this task, however their performance has been subpar compared to handcrafted techniques, especially while dealing with reverse loops. In this paper, we introduce the novel LCDNet that effectively detects loop closures in LiDAR point clouds by simultaneously identifying previously visited places and estimating the 6-DoF relative transformation between the current sca…
Multi-beam LiDAR sensors are increasingly used in robotics, particularly with autonomous cars for localization and perception tasks, both relying on the ability to build a precise map of the environment. For this, we propose a new real-time LiDAR-only odometry method called CT-ICP (for Continuous-Time ICP), completed into a full SLAM with a novel loop detection procedure. The core of this method, is the introduction of the combined continuity in the scan matching, and discontinuity between scans. It allows both the elastic distortion of the scan during the registration for increased precision,…
Shanghai Polytechnic University（上海理工大学）；Singapore Management University（新加坡管理学院）；Nanjing University of Science and Technology（南京理工大学）；University of Macau（澳门大学）
Edinburgh Centre for Robotics（爱丁堡机器人中心）；Heriot-Watt University（赫瑞-瓦特大学）
Numerous Simultaneous Localization and Mapping (SLAM) algorithms have been presented in last decade using different sensor modalities. However, robust SLAM in extreme weather conditions is still an open research problem. In this paper, RadarSLAM, a full radar based graph SLAM system, is proposed for reliable localization and mapping in large-scale environments. It is composed of pose tracking, local mapping, loop closure detection and pose graph optimization, enhanced by novel feature matching and probabilistic point cloud generation on radar images. Extensive experiments are conducted on a pu…
