We present a LiDAR-based and real-time capable 3D perception system for automated driving in urban domains. The hierarchical system design is able to model stationary and movable parts of the environment simultaneously and under real-time conditions. Our approach extends the state of the art by innovative in-detail enhancements for perceiving road users and drivable corridors even in case of non-flat ground surfaces and overhanging or protruding elements. We describe a runtime-efficient pointcloud processing pipeline, consisting of adaptive ground surface estimation, 3D clustering and motion classification stages. Based on the pipeline's output, the stationary environment is represented in a multi-feature mapping and fusion approach. Movable elements are represented in an object tracking system capable of using multiple reference points to account for viewpoint changes. We further enhance the tracking system by explicit consideration of occlusion and ambiguity cases. Our system is evaluated using a subset of the TUBS Road User Dataset. We enhance common performance metrics by considering application-driven aspects of real-world traffic scenarios. The perception system shows impressive results and is able to cope with the addressed scenarios while still preserving real-time capability.
Autonomous vehicles rely on their perception systems to acquire information about their immediate surroundings. It is necessary to detect the presence of other vehicles, pedestrians and other relevant entities. Safety concerns and the need for accurate estimations have led to the introduction of Light Detection and Ranging (LiDAR) systems in complement to the camera or radar-based perception systems. This article presents a review of state-of-the-art automotive LiDAR technologies and the perception algorithms used with those technologies. LiDAR systems are introduced first by analyzing the mai…
Oxford Robotics Institute（牛津机器人研究所）；University of Oxford（牛津大学）
Localization is a key challenge in many robotics applications. In this work we explore LIDAR-based global localization in both urban and natural environments and develop a method suitable for online application. Our approach leverages efficient deep learning architecture capable of learning compact point cloud descriptors directly from 3D data. The method uses an efficient feature space representation of a set of segmented point clouds to match between the current scene and the prior map. We show that down-sampling in the inner layers of the network can significantly reduce computation time wi…
ROS 2-Based LiDAR Perception Framework for Mobile Robots in Dynamic Production Environments, Utilizing Synthetic Data Generation, Transformation-Equivariant 3D Detection and Multi-Object Tracking
Chair of Intelligence in Quality Sensing (IQS), Laboratory for Machine Tools and Production Engineering (WZL), RWTH Aachen University（亚琛工业大学机床与生产工程实验室质量感知智能教席）；The Sirindhorn International Thai-German Graduate School of Engineering (TGGS), King Mongkut’s University of Technology North Bangkok (KMUTNB)（北曼谷先皇技术大学泰德国际工程研究生院）
Safe autonomous landing in urban cities is a necessity for the growing Unmanned Aircraft Systems (UAS) industry. In urgent situations, building rooftops, particularly flat rooftops, can provide local safe landing zones for small UAS. This paper investigates the real-time identification and selection of safe landing zones on rooftops based on LiDAR and camera sensor feedback. A visual high fidelity simulated city is constructed in the Unreal game engine, with particular attention paid to accurately generating rooftops and the common obstructions found thereon, e.g., ac units, water towers, air …
PERAL: Perception-Aware Motion Control for Passive LiDAR Excitation in Spherical Robots
Centre for Advanced Robotics Technology Innovation (CARTIN), School of Electrical and Electronic Engineering, Nanyang Technological University（先进机器人技术创新中心（CARTIN）、电气与电子工程学院、南洋理工大学）
The Hong Kong University of Science and Technology（香港科技大学）
Multiple LiDARs have progressively emerged on autonomous vehicles for rendering a wide field of view and dense measurements. However, the lack of precise calibration negatively affects their potential applications in localization and perception systems. In this paper, we propose a novel system that enables automatic multi-LiDAR calibration without any calibration target, prior environmental information, and initial values of the extrinsic parameters. Our approach starts with a hand-eye calibration for automatic initialization by aligning the estimated motions of each sensor. The resulting para…
Localization is a crucial capability for mobile robots and autonomous cars. In this paper, we address learning an observation model for Monte-Carlo localization using 3D LiDAR data. We propose a novel, neural network-based observation model that computes the expected overlap of two 3D LiDAR scans. The model predicts the overlap and yaw angle offset between the current sensor reading and virtual frames generated from a pre-built map. We integrate this observation model into a Monte-Carlo localization framework and tested it on urban datasets collected with a car in different seasons. The experi…
Due to their ubiquity and long-term stability, pole-like objects are well suited to serve as landmarks for vehicle localization in urban environments. In this work, we present a complete mapping and long-term localization system based on pole landmarks extracted from 3-D lidar data. Our approach features a novel pole detector, a mapping module, and an online localization module, each of which are described in detail, and for which we provide an open-source implementation at www.github.com/acschaefer/polex. In extensive experiments, we demonstrate that our method improves on the state of the ar…
Robust perception is an essential component to enable long-term operation of mobile robots. It depends on failure resilience through reliable sensor data and preprocessing, as well as failure awareness through introspection, for example the ability to self-assess localization performance. This paper presents CorAl: a principled, intuitive, and generalizable method to measure the quality of alignment between pairs of point clouds, which learns to detect alignment errors in a self-supervised manner. CorAl compares the differential entropy in the point clouds separately with the entropy in their …
Australian Center for Field Robotics, the University of Sydney（悉尼大学澳大利亚野外机器人中心）
Centimeter level globally accurate and consistent maps for autonomous vehicles navigation has long been achieved by on board real-time kinematic(RTK)-GPS in open areas. However when dealing with urban environments, GPS will experience multipath and blockage in urban canyon, under bridges, inside tunnels and in underground environments. In this paper we present strategies to efficiently register local maps in geographical coordinate systems through the tactical integration of GPS and information extracted from precisely geo-referenced high resolution aerial orthogonal imagery. Dense lidar point…
Modern autonomous vehicles rely heavily on mechanical LiDARs for perception. Current perception methods generally require 360° point clouds, collected sequentially as the LiDAR scans the azimuth and acquires consecutive wedge-shaped slices. The acquisition latency of a full scan (~ 100ms) may lead to outdated perception which is detrimental to safe operation. Recent streaming perception works proposed directly processing LiDAR slices and compensating for the narrow field of view (FOV) of a slice by reusing features from preceding slices. These works, however, are all based on a single modality…
University of Waterloo（滑铁卢大学）；University of Toronto（多伦多大学）
We introduce the Precise Synthetic Image and LiDAR (PreSIL) dataset for autonomous vehicle perception. Grand Theft Auto V (GTA V), a commercial video game, has a large detailed world with realistic graphics, which provides a diverse data collection environment. Existing works creating synthetic LiDAR data for autonomous driving with GTA V have not released their datasets, rely on an in-game raycasting function which represents people as cylinders, and can fail to capture vehicles past 30 metres. Our work creates a precise LiDAR simulator within GTA V which collides with detailed models for all…
Ford AV LLC（福特自动驾驶有限责任公司）；Ford Motor Company（福特汽车公司）
This paper presents a localization technique using aerial imagery maps and LIDAR based ground reflectivity for autonomous vehicles in urban environments. Traditional localization techniques using LIDAR reflectivity rely on high definition reflectivity maps generated from a mapping vehicle. The cost and effort required to maintain such prior maps are generally very high because it requires a fleet of expensive mapping vehicles. In this work we propose a localization technique where the vehicle localizes using aerial/satellite imagery, eradicating the need to develop and maintain complex high-de…
Hong Kong Polytechnic University（香港理工大学）；University of California, Berkeley（加州大学伯克利分校）
Mapping and localization is a critical module of autonomous driving, and significant achievements have been reached in this field. Beyond Global Navigation Satellite System (GNSS), research in point cloud registration, visual feature matching, and inertia navigation has greatly enhanced the accuracy and robustness of mapping and localization in different scenarios. However, highly urbanized scenes are still challenging: LIDAR- and camera-based methods perform poorly with numerous dynamic objects; the GNSS-based solutions experience signal loss and multipath problems; the inertia measurement un…
University of California, Berkeley（加利福尼亚大学伯克利分校）；Ulm University（乌尔姆大学）
Detecting dynamic objects and predicting static road information such as drivable areas and ground heights are crucial for safe autonomous driving. Previous works studied each perception task separately, and lacked a collective quantitative analysis. In this work, we show that it is possible to perform all perception tasks via a simple and efficient multi-task network. Our proposed network, LidarMTL, takes raw LiDAR point cloud as inputs, and predicts six perception outputs for 3D object detection and road understanding. The network is based on an encoder-decoder architecture with 3D sparse co…
DeepIPCv2: LiDAR-powered Robust Environmental Perception and Navigational Control for Autonomous Vehicle
Department of Computer Science and Electronics, Universitas Gadjah Mada（计算机科学与电子系，加查马达大学）；Department of Computer Science and Engineering, Toyohashi University of Technology（计算机科学与工程系，toyohashi技术大学）
Carnegie Mellon University（卡内基梅隆大学）；Amazon（亚马逊）；University of Southern California（南加州大学）
The past few years have witnessed an increasing interest in improving the perception performance of LiDARs on autonomous vehicles. While most of the existing works focus on developing new deep learning algorithms or model architectures, we study the problem from the physical design perspective, i.e., how different placements of multiple LiDARs influence the learning-based perception. To this end, we introduce an easy-to-compute information-theoretic surrogate metric to quantitatively and fast evaluate LiDAR placement for 3D detection of different types of objects. We also present a new data co…
A Unified Generative Framework for Realistic Lidar Simulation in Autonomous Driving Systems
WMG, University of Warwick（华威大学WMG学院）；School of Electronics, Electrical Engineering and Computer Science at Queen’s University of Belfast（贝尔法斯特女王大学电子、电气工程与计算机科学学院）
Light Detection and Ranging (LiDAR) based Simultaneous Localization and Mapping (SLAM) has drawn increasing interests in autonomous driving. However, LiDAR-SLAM suffers from accumulating errors which can be significantly mitigated by Global Navigation Satellite System (GNSS). Precise Point Positioning (PPP), an accurate GNSS operation mode independent of base stations, gains more popularity in unmanned systems. Considering the features of the two technologies, LiDAR-SLAM and PPP, this paper proposes a SLAM system, namely P3-LOAM (PPP based LiDAR Odometry and Mapping) which couples LiDAR-SLAM a…
Using publicly accessible maps, we propose a novel vehicle localization method that can be applied without using prior light detection and ranging (LiDAR) maps. Our method generates OSM descriptors by calculating the distances to buildings from a location in OpenStreetMap at a regular angle, and LiDAR descriptors by calculating the shortest distances to building points from the current location at a regular angle. Comparing the OSM descriptors and LiDAR descriptors yields a highly accurate vehicle localization result. Compared to methods that use prior LiDAR maps, our method presents two main …
Robust and precise localization is essential for the autonomous system with navigation requirements. Light detection and ranging (LiDAR) odometry is extensively studied in the past decades to achieve this goal. Satisfactory accuracy can be achieved in scenarios with abundant environmental features using existing LiDAR odometry (LO) algorithms. Unfortunately, the performance of the LiDAR odometry is significantly degraded in urban canyons with numerous dynamic objects and complex environmental structures. Meanwhile, it is still not clear from the existing literature which LO algorithms perform …
Department of Civil and Environmental Engineering, KAIST（韩国科学技术院（KAIST）土木与环境工程系）
This paper presents a Light Detection and Ranging (LiDAR) data set that targets complex urban environments. Urban environments with high-rise buildings and congested traffic pose a significant challenge for many robotics applications. The presented data set is unique in the sense it is able to capture the genuine features of an urban environment (e.g. metropolitan areas, large building complexes and underground parking lots). Data of two-dimensional (2D) and threedimensional (3D) LiDAR, which are typical types of LiDAR sensors, are provided in the data set. The two 16-ray 3D LiDARs are tilted …
The Hong Kong University of Science and Technology（香港科技大学）；Chinese Academy of Sciences（中国科学院）
High precision 3D LiDARs are still expensive and hard to acquire. This paper presents the characteristics of RS-LiDAR, a model of low-cost LiDAR with sufficient supplies, in comparison with VLP-16. The paper also provides a set of evaluations to analyze the characterizations and performances of LiDARs sensors. This work analyzes multiple properties, such as drift effects, distance effects, color effects and sensor orientation effects, in the context of 3D perception. By comparing with Velodyne LiDAR, we found RS-LiDAR as a cheaper and acquirable substitute of VLP-16 with similar efficiency.
GNSS and LiDAR odometry are complementary as they provide absolute and relative positioning, respectively. Their integration in a loosely-coupled manner is straightforward but is challenged in urban canyons due to the GNSS signal reflections. Recent proposed 3D LiDAR-aided (3DLA) GNSS methods employ the point cloud map to identify the non-line-of-sight (NLOS) reception of GNSS signals. This facilitates the GNSS receiver to obtain improved urban positioning but not achieve a sub-meter level. GNSS real-time kinematics (RTK) uses carrier phase measurements to obtain decimeter-level positioning. I…
Robust and lane-level positioning is essential for autonomous vehicles. As an irreplaceable sensor, LiDAR can provide continuous and high-frequency pose estimation by means of mapping, on condition that enough environment features are available. The error of mapping can accumulate over time. Therefore, LiDAR is usually integrated with other sensors. In diverse urban scenarios, the environment feature availability relies heavily on the traffic (moving and static objects) and the degree of urbanization. Common LiDAR-based SLAM demonstrations tend to be studied in light traffic and less urbanized…
Reliable and accurate localization is crucial for mobile autonomous systems. Pole-like objects, such as traffic signs, poles, lamps, etc., are ideal landmarks for localization in urban environments due to their local distinctiveness and long-term stability. In this paper, we present a novel, accurate, and fast pole extraction approach that runs online and has little computational demands such that this information can be used for a localization system. Our method performs all computations directly on range images generated from 3D LiDAR scans, which avoids processing 3D point cloud explicitly …
In this paper, we propose a 3D LiDAR aided global navigation satellite system (GNSS) non-line-of-sight (NLOS) mitigation method caused by both static buildings and dynamic objects. A sliding window map describing the surrounding of the ego-vehicle is first generated, based on real-time 3D point clouds from a 3D LiDAR sensor. Then, NLOS receptions are detected based on the sliding window map using a proposed fast searching method which is free of the initial guess of the position of the GNSS receiver. Instead of directly excluding the detected NLOS satellites from further positioning estimation…
Matched Filtering based LiDAR Place Recognition for Urban and Natural Environments
Queensland University of Technology（昆士兰科技大学）；QUT Centre for Robotics（昆士兰科技大学机器人中心）；School of Electrical Engineering and Robotics（电气工程与机器人学院）；Intel Labs（英特尔实验室）
HDA-LVIO: A High-Precision LiDAR-Visual-Inertial Odometry in Urban Environments with Hybrid Data Association
College of Intelligent Systems Science and Engineering, Harbin Engineering University（哈尔滨工程大学智能系统科学与工程学院）；National Innovation Institute of Defense Technology, Academy of Military Sciences China（中国军事科学院国防科技创新研究院）；Tianjin Artificial Intelligence Innovation Center (TAIIC)（天津人工智能创新中心）
ParisLuco3D: A high-quality target dataset for domain generalization of LiDAR perception
Centre for Robotics, Mines Paris - PSL, PSL University（巴黎文理研究大学、巴黎矿业学院机器人中心）；U2IS, ENSTA Paris, Institut Polytechnique de Paris（巴黎综合理工学院、巴黎国立高等先进技术学院U2IS实验室）
HumanoidPano: Hybrid Spherical Panoramic-LiDAR Cross-Modal Perception for Humanoid Robots
Beijing Innovation Center of Humanoid Robotics（北京人形机器人创新中心）；Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；Hong Kong University of Science and Technology（香港科技大学）；Beijing Institute of Technology（北京理工大学）；ETH Zurich（苏黎世联邦理工学院）；Nanyang Technological University（南洋理工大学）；X-Humanoid（人形智能科技公司）
OPPO US Research Center（OPPO美国研究中心）；Cornell University（康奈尔大学）
This paper presents a novel framework for planning in unknown and occluded urban spaces. We specifically focus on turns and intersections where occlusions significantly impact navigability. Our approach uses an inpainting model to fill in a sparse, occluded, semantic lidar point cloud and plans dynamically feasible paths for a vehicle to traverse through the open and inpainted spaces. We demonstrate our approach using a car's lidar data with real-time occlusions, and show that by inpainting occluded areas, we can plan longer paths, with more turn options compared to without inpainting; in addi…
State Key Laboratory of Internet of Things for Smart City (SKL-IOTSC)（物联网智能城市国家重点实验室）；Faculty of Science and Technology（科技学院）；University of Macau（澳门大学）；School of Computing and Information Systems（计算与信息学院）；Singapore Management University（新加坡管理大学）；Department of Naval Architecture and Marine Engineering and Department of Robotics（船舶工程与海洋工程系和机器人系）；University of Michigan（密歇根大学）；Department of Robotics（机器人系）；Mohamed bin Zayed University of Artificial Intelligence (MBZUAI)（穆罕默德·本·拉希德智能人工智能大学）
Multi-LiDAR Localization and Mapping Pipeline for Urban Autonomous Driving
Institute of Automotive Technology, Munich Institute of Robotics and Machine Intelligence (MIRMI), Technical University of Munich（慕尼黑工业大学汽车技术研究所，慕尼黑机器人与机器智能研究所）；Professorship Autonomous Vehicle Systems, Munich Institute of Robotics and Machine Intelligence (MIRMI), Technical University of Munich（慕尼黑工业大学自动驾驶车辆系统教授席，慕尼黑机器人与机器智能研究所）
PRISM-Loc: a Lightweight Long-range LiDAR Localization in Urban Environments with Topological Maps
Federal Research Center for Computer Science and Control of Russian Academy of Sciences（俄罗斯科学院计算机科学与控制联邦研究中心）；Moscow Institute of Physics and Technology (MIPT)（莫斯科物理技术学院）；Artificial Intelligence Research Institute (AIRI)（人工智能研究所）；Innopolis University（Innopolis大学）
Aalto University（阿尔托大学）；University of Bonn（波恩大学）；University of Oxford（牛津大学）
Robust and accurate localization is a basic requirement for mobile autonomous systems. Pole-like objects, such as traffic signs, poles, and lamps are frequently used landmarks for localization in urban environments due to their local distinctiveness and long-term stability. In this paper, we present a novel, accurate, and fast pole extraction approach based on geometric features that runs online and has little computational demands. Our method performs all computations directly on range images generated from 3D LiDAR scans, which avoids processing 3D point clouds explicitly and enables fast po…
