Place recognition is an important component for autonomous vehicles to achieve loop closing or global localization. In this paper, we tackle the problem of place recognition based on sequential 3D LiDAR scans obtained by an onboard LiDAR sensor. We propose a transformer-based network named SeqOT to exploit the temporal and spatial information provided by sequential range images generated from the LiDAR data. It uses multi-scale transformers to generate a global descriptor for each sequence of LiDAR range images in an end-to-end fashion. During online operation, our SeqOT finds similar places by matching such descriptors between the current query sequence and those stored in the map. We evaluate our approach on four datasets collected with different types of LiDAR sensors in different environments. The experimental results show that our method outperforms the state-of-the-art LiDAR-based place recognition methods and generalizes well across different environments. Furthermore, our method operates online faster than the frame rate of the sensor. The implementation of our method is released as open source at: https://github.com/BIT-MJY/SeqOT.
A Survey on Global LiDAR Localization: Challenges, Advances and Open Problems
Hong Kong University of Science and Technology（香港科技大学）；Zhejiang University（浙江大学）；National University of Defense Technology（国防科技大学）；University of Bonn（波恩大学）
LIO-EKF: High Frequency LiDAR-Inertial Odometry using Extended Kalman Filters
Institute of Geodesy and Geoinformation, University of Bonn（波恩大学大地测量与地理信息研究所）；Department of Engineering Science at the University of Oxford（牛津大学工程科学系）；Lamarr Institute for Machine Learning and Artificial Intelligence（拉马尔机器学习与人工智能研究所）
This paper deals with the development of a localization methodology for autonomous vehicles using only a $3\Dim$ LIDAR sensor. In the context of this paper, localizing a vehicle in a known 3D global map of the environment is essentially to find its global $3\Dim$ pose (position and orientation) within this map. The problem of tracking is then to use sequential LIDAR scan measurement to also estimate other states such as velocity and angular rates, in addition to the global pose of the vehicle. Particle filters are often used in localization and tracking, as in applications of simultaneously lo…
BEVPlace++: Fast, Robust, and Lightweight LiDAR Global Localization for Unmanned Ground Vehicles
College of Intelligence Science and Technology, National University of Defense Technology（智能科学与技术学院，国防科技大学）；Haomo.AI Technology Co., Ltd（哈莫人工智能技术有限公司）；Ningbo Innovation Center, Zhejiang University（宁波创新中心，浙江大学）；College of Instrument Science and Optoelectronics Engineering, Beihang University（仪器科学与光电工程学院，北航）；College of Information Science and Electronic Engineering, Zhejiang University（信息科学与电子工程学院，浙江大学）
RING#: PR-by-PE Global Localization with Roto-translation Equivariant Gram Learning
Institute of Cyber-Systems and Control, Zhejiang University（浙江大学控制科学与工程学院（ cyber系统与控制研究所））；School of Astronautics, Beihang University（北京航空航天大学宇航学院）；College of Intelligence Science and Technology, National University of Defense Technology（国防科技大学智能科学学院）
Continuous robot operation in extreme scenarios such as underground mines or sewers is difficult because exteroceptive sensors may fail due to fog, darkness, dirt or malfunction. So as to enable autonomous navigation in these kinds of situations, we have developed a type of proprioceptive localization which exploits the foot contacts made by a quadruped robot to localize against a prior map of an environment, without the help of any camera or LIDAR sensor. The proposed method enables the robot to accurately re-localize itself after making a sequence of contact events over a terrain feature. Th…
Map construction in large scale outdoor environment is of importance for robots to robustly fulfill their tasks. Massive sessions of data should be merged to distinguish low dynamics in the map, which otherwise might debase the performance of localization and navigation algorithms. In this paper we propose a method for multi-session map construction in large scale outdoor environment using 3D LiDAR. To efficiently align the maps from different sessions, a laser-based loop closure detection method is integrated and the sequential information within the submaps is utilized for higher robustness.…
Reliable-loc: Robust sequential LiDAR global localization in large-scale street scenes based on verifiable cues
State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing, Wuhan University（武汉大学测绘遥感信息工程国家重点实验室）；School of Electrical and Electronic Engineering, Nanyang Technological University（南洋理工大学电气与电子工程学院）；School of Earth Sciences and Engineering, Hohai University（河海大学地球科学与工程学院）；Hubei Luojia Laboratory（湖北珞珈实验室）
From Single Scan to Sequential Consistency: A New Paradigm for LIDAR Relocalization
Fujian Key Laboratory of Sensing and Computing for Smart Cities, Xiamen University（厦门大学福建省智慧城市感知与计算重点实验室）；Ministry of Education of China, Xiamen University（厦门大学（中国教育部））
SLCF-Net: Sequential LiDAR-Camera Fusion for Semantic Scene Completion using a 3D Recurrent U-Net
University of Bonn（波恩大学）；Lamarr Institute for Machine Learning and Artificial Intelligence（拉马尔机器学习与人工智能研究所）；Center for Robotics（机器人中心）；Computer Science Institute VI – Intelligent Systems and Robotics（计算机科学研究所VI——智能系统与机器人）
Opel Automobile GmbH（欧宝汽车公司）；Institute of Measurement and Control Systems, Karlsruhe Institute of Technology (KIT)（卡尔斯鲁厄理工学院测量与控制系统研究所）
3D pedestrian detection is a challenging task in automated driving because pedestrians are relatively small, frequently occluded and easily confused with narrow vertical objects. LiDAR and camera are two commonly used sensor modalities for this task, which should provide complementary information. Unexpectedly, LiDAR-only detection methods tend to outperform multisensor fusion methods in public benchmarks. Recently, PointPainting has been presented to eliminate this performance drop by effectively fusing the output of a semantic segmentation network instead of the raw image information. In thi…
Most scanning LiDAR sensors generate a sequence of point clouds in real-time. While conventional 3D object detectors use a set of unordered LiDAR points acquired over a fixed time interval, recent studies have revealed that substantial performance improvement can be achieved by exploiting the spatio-temporal context present in a sequence of LiDAR point sets. In this paper, we propose a novel 3D object detection architecture, which can encode LiDAR point cloud sequences acquired by multiple successive scans. The encoding process of the point cloud sequence is performed on two different time sca…
Enhancing Lidar-based Object Detection in Adverse Weather using Offset Sequences in Time
Institute for Automotive Engineering RWTH Aachen University（亚琛工业大学汽车工程研究所）；Institute of Imaging and Computer Vision RWTH Aachen University（亚琛工业大学图像与计算机视觉研究所）
We propose an algorithm to (i) learn online a deep signed distance function (SDF) with a LiDAR-equipped robot to represent the 3D environment geometry, and (ii) plan collision-free trajectories given this deep learned map. Our algorithm takes a stream of incoming LiDAR scans and continually optimizes a neural network to represent the SDF of the environment around its current vicinity. When the SDF network quality saturates, we cache a copy of the network, along with a learned confidence metric, and initialize a new SDF network to continue mapping new regions of the environment. We then concate…
Robust real-time detection and motion forecasting of traffic participants is necessary for autonomous vehicles to safely navigate urban environments. In this paper, we present RV-FuseNet, a novel end-to-end approach for joint detection and trajectory estimation directly from time-series LiDAR data. Instead of the widely used bird's eye view (BEV) representation, we utilize the native range view (RV) representation of LiDAR data. The RV preserves the full resolution of the sensor by avoiding the voxelization used in the BEV. Furthermore, RV can be processed efficiently due to its compactness. P…
Accurate moving object segmentation is an essential task for autonomous driving. It can provide effective information for many downstream tasks, such as collision avoidance, path planning, and static map construction. How to effectively exploit the spatial-temporal information is a critical question for 3D LiDAR moving object segmentation (LiDAR-MOS). In this work, we propose a novel deep neural network exploiting both spatial-temporal information and different representation modalities of LiDAR scans to improve LiDAR-MOS performance. Specifically, we first use a range image-based dual-branch …
SNAIL Radar: A large-scale diverse benchmark for evaluating 4D-radar-based SLAM
Wuhan University（武汉大学）；The state key Lab of Information Engineering in Surveying, Mapping And Remote Sensing (LIESMARS), Wuhan University（武汉大学测绘遥感信息工程国家重点实验室）；Wuhan Quantum Institute（武汉量子研究院）
Camera and lidar are important sensor modalities for robotics in general and self-driving cars in particular. The sensors provide complementary information offering an opportunity for tight sensor-fusion. Surprisingly, lidar-only methods outperform fusion methods on the main benchmark datasets, suggesting a gap in the literature. In this work, we propose PointPainting: a sequential fusion method to fill this gap. PointPainting works by projecting lidar points into the output of an image-only semantic segmentation network and appending the class scores to each point. The appended (painted) poin…
In this work, we propose \textit{MVFuseNet}, a novel end-to-end method for joint object detection and motion forecasting from a temporal sequence of LiDAR data. Most existing methods operate in a single view by projecting data in either range view (RV) or bird's eye view (BEV). In contrast, we propose a method that effectively utilizes both RV and BEV for spatio-temporal feature learning as part of a temporal fusion network as well as for multi-scale feature learning in the backbone network. Further, we propose a novel sequential fusion approach that effectively utilizes multiple views in the …
Most real-world 3D sensors such as LiDARs perform fixed scans of the entire environment, while being decoupled from the recognition system that processes the sensor data. In this work, we propose a method for 3D object recognition using light curtains, a resource-efficient controllable sensor that measures depth at user-specified locations in the environment. Crucially, we propose using prediction uncertainty of a deep learning based 3D point cloud detector to guide active perception. Given a neural network's uncertainty, we derive an optimization objective to place light curtains using the pr…
Tokyo Institute of Technology（东京工业大学）；National Institute of Advanced Industrial Science and Technology（产业技术综合研究所）
In this paper, we propose an automatic labeled sequential data generation pipeline for human segmentation and velocity estimation with point clouds. Considering the impact of deep neural networks, state-of-the-art network architectures have been proposed for human recognition using point clouds captured by Light Detection and Ranging (LiDAR). However, one disadvantage is that legacy datasets may only cover the image domain without providing important label information and this limitation has disturbed the progress of research to date. Therefore, we develop an automatic labeled sequential data …
FlowCalib: LiDAR-to-Vehicle Miscalibration Detection using Scene Flows
TUM School of Engineering and Design, Chair of Automotive Technology, Technical University of Munich（慕尼黑技术大学工程与设计学院，汽车技术教授职位）；TUM School of Computation, Information and Technology, Technical University of Munich（慕尼黑技术大学计算、信息与技术学院）
U4D: Uncertainty-Aware 4D World Modeling from LiDAR Sequences
Nanjing University of Aeronautics and Astronautics（南京航空航天大学）；National University of Singapore（新加坡国立大学）；Fudan University（复旦大学）；S-Lab, Nanyang Technological University（南洋理工大学S实验室）；Nanjing University of Posts and Telecommunications（南京邮电大学）；SKL-TI
Accurate Measurement of 3D and 2D Circular Centers With Application to LiDAR-Camera Extrinsic Calibration
Thrust of Robotics and Autonomous Systems, The Hong Kong University of Science and Technology (Guangzhou)（机器人与自主系统研究所，香港科技大学（广州））；Lower Airspace Economy Research Institute, International Digital Economy Academy, ShenZhen（空域经济研究所，国际数字经济学院）；Horizon-Continental Technology Corporation（Horizon-Continental技术公司）；State Key Laboratory of Rail Traffic Control and Safety and the Beijing Engineering Research Center of EMC and GNSS Technology for Rail Transportation, School of Electronic and Information Engineering, Beijing Jiaotong University（轨道交通控制与安全国家重点实验室，北京交通大学电子信息工程学院，电磁兼容与GNSS技术轨道交通工程研究中心）
Compared to the onboard camera and laser scanner, radar sensor provides lighting and weather invariant sensing, which is naturally suitable for long-term localization under adverse conditions. However, radar data is sparse and noisy, resulting in challenges for radar mapping. On the other hand, the most popular available map currently is built by lidar. In this paper, we propose an end-to-end deep learning framework for Radar Localization on Lidar Map (RaLL) to bridge the gap, which not only achieves the robust radar localization but also exploits the mature lidar mapping technique, thus reduc…
This paper presents a novel indoor layout estimation system based on the fusion of 2D LiDAR and intensity camera data. A ground robot explores an indoor space with a single floor and vertical walls, and collects a sequence of intensity images and 2D LiDAR datasets. The LiDAR provides accurate depth information, while the camera captures high-resolution data for semantic interpretation. The alignment of sensor outputs and image segmentation are computed jointly by aligning LiDAR points, as samples of the room contour, to ground-wall boundaries in the images. The alignment problem is decoupled i…
Autonomous vehicles operate in a dynamic environment, where the speed with which a vehicle can perceive and react impacts the safety and efficacy of the system. LiDAR provides a prominent sensory modality that informs many existing perceptual systems including object detection, segmentation, motion estimation, and action recognition. The latency for perceptual systems based on point cloud data can be dominated by the amount of time for a complete rotational scan (e.g. 100 ms). This built-in data capture latency is artificial, and based on treating the point cloud as a camera image in order to …
Chalmers University of Technology（查尔姆斯理工大学）；University of Gothenburg（哥德堡大学）
This paper presents a dataset, called Reeds, for research on robot perception algorithms. The dataset aims to provide demanding benchmark opportunities for algorithms, rather than providing an environment for testing application-specific solutions. A boat was selected as a logging platform in order to provide highly dynamic kinematics. The sensor package includes six high-performance vision sensors, two long-range lidars, radar, as well as GNSS and an IMU. The spatiotemporal resolution of sensors were maximized in order to provide large variations and flexibility in the data, offering evaluati…
