With the recent boost in autonomous driving, increased attention has been paid on radars as an input for occupancy mapping. Besides their many benefits, the inference of occupied space based on radar detections is notoriously difficult because of the data sparsity and the environment dependent noise (e.g. multipath reflections). Recently, deep learning-based inverse sensor models, from here on called deep ISMs, have been shown to improve over their geometric counterparts in retrieving occupancy information. Nevertheless, these methods perform a data-driven interpolation which has to be verified later on in the presence of measurements. In this work, we describe a novel approach to integrate deep ISMs together with geometric ISMs into the evidential occupancy mapping framework. Our method leverages both the capabilities of the data-driven approach to initialize cells not yet observable for the geometric model effectively enhancing the perception field and convergence speed, while at the same time use the precision of the geometric ISM to converge to sharp boundaries. We further define a lower limit on the deep ISM estimate's certainty together with analytical proofs of convergence which we use to distinguish cells that are solely allocated by the deep ISM from cells already verified using the geometric approach.
FZI Research Center for Information Technology（FZI信息技术研究中心）；Karlsruhe Institute of Technology (KIT)（卡尔斯鲁厄理工学院（KIT））
A detailed environment representation is a crucial component of automated vehicles. Using single range sensor scans, data is often too sparse and subject to occlusions. Therefore, we present a method to augment occupancy grid maps from single views to be similar to evidential occupancy maps acquired from different views using Deep Learning. To accomplish this, we estimate motion between subsequent range sensor measurements and create an evidential 3D voxel map in an extensive post-processing step. Within this voxel map, we explicitly model uncertainty using evidence theory and create a 2D proj…
Occupancy mapping has been widely utilized to represent the surroundings for autonomous robots to perform tasks such as navigation and manipulation. While occupancy mapping in 2-D environments has been well-studied, there have been few approaches suitable for 3-D dynamic occupancy mapping which is essential for aerial robots. This paper presents a novel 3-D dynamic occupancy mapping algorithm called DS-K3DOM. We first establish a Bayesian method to sequentially update occupancy maps for a stream of measurements based on the random finite set theory. Then, we approximate it with particles in th…
Ulm University（乌尔姆大学）；Robert Bosch GmbH（罗伯特·博世有限公司）；Otto von Guericke University Magdeburg（马格德堡奥托·冯·格里克大学）
A common approach for modeling the environment of an autonomous vehicle are dynamic occupancy grid maps, in which the surrounding is divided into cells, each containing the occupancy and velocity state of its location. Despite the advantage of modeling arbitrary shaped objects, the used algorithms rely on hand-designed inverse sensor models and semantic information is missing. Therefore, we introduce a multi-task recurrent neural network to predict grid maps providing occupancies, velocity estimates, semantic information and the driveable area. During training, our network architecture, which …
Evidential occupancy grid maps (OGMs) are a popular representation of the environment of automated vehicles. Inverse sensor models (ISMs) are used to compute OGMs from sensor data such as lidar point clouds. Geometric ISMs show a limited performance when estimating states in unobserved but inferable areas and have difficulties dealing with ambiguous input. Deep learning-based ISMs face the challenge of limited training data and they often cannot handle uncertainty quantification yet. We propose a deep learning-based framework for learning an OGM algorithm which is both capable of quantifying f…
Radar presents a promising alternative to lidar and vision in autonomous vehicle applications, able to detect objects at long range under a variety of weather conditions. However, distinguishing between occupied and free space from raw radar power returns is challenging due to complex interactions between sensor noise and occlusion. To counter this we propose to learn an Inverse Sensor Model (ISM) converting a raw radar scan to a grid map of occupancy probabilities using a deep neural network. Our network is self-supervised using partial occupancy labels generated by lidar, allowing a robot to…
Stanford University（斯坦福大学）；University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
Autonomous vehicles must reason about spatial occlusions in urban environments to ensure safety without being overly cautious. Prior work explored occlusion inference from observed social behaviors of road agents, hence treating people as sensors. Inferring occupancy from agent behaviors is an inherently multimodal problem; a driver may behave similarly for different occupancy patterns ahead of them (e.g., a driver may move at constant speed in traffic or on an open road). Past work, however, does not account for this multimodality, thus neglecting to model this source of aleatoric uncertainty…
Karlsruhe Institute for Technology (KIT)（卡尔斯鲁厄理工学院）；Fraunhofer Institute of Optronics, System Technologies and Image Exploitation（弗劳恩霍夫光电、系统技术与图像利用研究所）；FZI Forschungszentrum Informatik（FZI信息学研究中心）
In this work we present a fast occupancy map building approach based on the VDB datastructure. Existing log-odds based occupancy mapping systems are often not able to keep up with the high point densities and framerates of modern sensors. Therefore, we suggest a highly optimized approach based on a modern datastructure coming from a computer graphic background. A multithreaded insertion scheme allows occupancy map building at unprecedented speed. Multiple optimizations allow for a customizable tradeoff between runtime and map quality. We first demonstrate the effectiveness of the approach quan…
Institute of Measurement, Control, and Microtechnology, Ulm University（乌尔姆大学测量、控制与微技术研究所）
Long-term situation prediction plays a crucial role in the development of intelligent vehicles. A major challenge still to overcome is the prediction of complex downtown scenarios with multiple road users, e.g., pedestrians, bikes, and motor vehicles, interacting with each other. This contribution tackles this challenge by combining a Bayesian filtering technique for environment representation, and machine learning as long-term predictor. More specifically, a dynamic occupancy grid map is utilized as input to a deep convolutional neural network. This yields the advantage of using spatially dis…
This paper describes a system whereby a robot detects and track human-meaningful navigational cues as it navigates in an indoor environment. It is intended as the sensor front-end for a mobile robot system that can communicate its navigational context with human users. From simulated LiDAR scan data we construct a set of 2D occupancy grid bitmaps, then hand-label these with human-scale navigational features such as closed doors, open corridors and intersections. We train a Convolutional Neural Network (CNN) to recognize these features on input bitmaps. In our demonstration system, these featur…
SWA-SOP: Spatially-aware Window Attention for Semantic Occupancy Prediction in Autonomous Driving
Autonomous Intelligent Systems group, Computer Science Institute VI – Intelligent Systems and Robotics, University of Bonn, Germany（bonn大学计算机科学研究所VI——智能系统与机器人组）；Center for Robotics, University of Bonn, Germany（bonn大学机器人中心）；Lamarr Institute for Machine Learning and Artificial Intelligence, Germany（德国lamarr机器学习与人工智能研究所）
Universidade Federal do Espírito Santo（圣灵联邦大学）；Instituto Federal do Espírito Santo（圣灵联邦研究所）
In this work, we present a novel strategy for correcting imperfections in occupancy grid maps called map decay. The objective of map decay is to correct invalid occupancy probabilities of map cells that are unobservable by sensors. The strategy was inspired by an analogy between the memory architecture believed to exist in the human brain and the maps maintained by an autonomous vehicle. It consists in merging sensory information obtained during runtime (online) with a priori data from a high-precision map constructed offline. In map decay, cells observed by sensors are updated using tradition…
Through constant improvements in recent years radar sensors have become a viable alternative to lidar as the main distancing sensor of an autonomous vehicle. Although robust and with the possibility to directly measure the radial velocity, it brings it's own set of challenges, for which existing algorithms need to be adapted. One core algorithm of a perception system is dynamic occupancy grid mapping, which has traditionally relied on lidar. In this paper we present a dual-weight particle filter as an extension for a Bayesian occupancy grid mapping framework to allow to operate it with radar a…
Occupancy grid maps (OGMs) are fundamental to most systems for autonomous robotic navigation. However, CPU-based implementations struggle to keep up with data rates from modern 3D lidar sensors, and provide little capacity for modern extensions which maintain richer voxel representations. This paper presents OHM, our open source, GPU-based OGM framework. We show how the algorithms can be mapped to GPU resources, resolving difficulties with contention to obtain a successful implementation. The implementation supports many modern OGM algorithms including NDT-OM, NDT-TM, decay-rate and TSDF. A th…
Mercedes-Benz RD/ASF（梅赛德斯-奔驰研发/自动驾驶安全基础部门）；University of Pennsylvania（宾夕法尼亚大学）；Fraunhofer FKIE（弗劳恩霍夫通信、信息处理和人机工程学研究所）
Grid maps are widely established for the representation of static objects in robotics and automotive applications. Though, incorporating velocity information is still widely examined because of the increased complexity of dynamic grids concerning both velocity measurement models for radar sensors and the representation of velocity in a grid framework. In this paper, both issues are addressed: sensor models and an efficient grid framework, which are required to ensure efficient and robust environment perception with radar. To that, we introduce new inverse radar sensor models covering radar sen…
Ford Werke GmbH（福特汽车公司（Ford Werke GmbH））；Institute for Automotive Engineering, RWTH Aachen University（亚琛工业大学汽车工程研究所（Institute for Automotive Engineering, RWTH Aachen University））
To perform high speed tasks, sensors of autonomous cars have to provide as much information in as few time steps as possible. However, radars, one of the sensor modalities autonomous cars heavily rely on, often only provide sparse, noisy detections. These have to be accumulated over time to reach a high enough confidence about the static parts of the environment. For radars, the state is typically estimated by accumulating inverse detection models (IDMs). We employ the recently proposed evidential convolutional neural networks which, in contrast to IDMs, compute dense, spatially coherent infer…
Evidential grids have recently shown interesting properties for mobile object perception. Evidential grids are a generalisation of Bayesian occupancy grids using Dempster- Shafer theory. In particular, these grids can handle efficiently partial information. The novelty of this article is to propose a perception scheme enhanced by geo-referenced maps used as an additional source of information, which is fused with a sensor grid. The paper presents the key stages of such a data fusion process. An adaptation of conjunctive combination rule is presented to refine the analysis of the conflicting in…
Center of Robotics（机器人中心）；Mines ParisTech（巴黎高等矿业学院）；PSL Research University（巴黎文理研究大学）；California PATH（加州PATH研究所）；University of California, Berkeley（加州大学伯克利分校）
Automation driving techniques have seen tremendous progresses these last years, particularly due to a better perception of the environment. In order to provide safe yet not too conservative driving in complex urban environment, data fusion should not only consider redundant sensing to characterize the surrounding obstacles, but also be able to describe the uncertainties and errors beyond presence/absence (be it binary or probabilistic). This paper introduces an enriched representation of the world, more precisely of the potential existence of obstacles through an evidential grid map. A method …
Johns Hopkins University Applied Physics Lab（约翰斯·霍普金斯大学应用物理实验室）；Johns Hopkins University（约翰斯·霍普金斯大学）
Safe and high-speed navigation is a key enabling capability for real world deployment of robotic systems. A significant limitation of existing approaches is the computational bottleneck associated with explicit mapping and the limited field of view (FOV) of existing sensor technologies. In this paper, we study algorithmic approaches that allow the robot to predict spaces extending beyond the sensor horizon for robust planning at high speeds. We accomplish this using a generative neural network trained from real-world data without requiring human annotated labels. Further, we extend our existin…
IBM T. J. Watson Research Center（IBM T. J. 沃森研究中心）
In this paper we provide an overview of a new framework for robot perception, real-world modelling, and navigation that uses a stochastic tesselated representation of spatial information called the Occupancy Grid. The Occupancy Grid is a multi-dimensional random field model that maintains probabilistic estimates of the occupancy state of each cell in a spatial lattice. Bayesian estimation mechanisms employing stochastic sensor models allow incremental updating of the Occupancy Grid using multi-view, multi-sensor data, composition of multiple maps, decision-making, and incorporation of robot an…
UMR CNRS 6599 Heudiasyc（法国国家科学研究中心6599联合研究单元Heudiasyc实验室）；University of Technology of Compiègne（贡比涅技术大学）
Evidential grids have been recently used for mobile object perception. The novelty of this article is to propose a perception scheme using prior map knowledge. A geographic map is considered an additional source of information fused with a grid representing sensor data. Yager's rule is adapted to exploit the Dempster-Shafer conflict information at large. In order to distinguish stationary and mobile objects, a counter is introduced and used as a factor for mass function specialisation. Contextual discounting is used, since we assume that different pieces of information become obsolete at diffe…
Real-time parking occupancy information is valuable for guiding drivers' searching for parking spaces. Recently many parking detection systems using range-based on-vehicle sensors are invented, but they disregard the practical difficulty of obtaining access to raw sensory data which are required for any feature-based algorithm. In this paper, we focus on a system using short-range radars (SRR) embedded in Advanced Driver Assistance System (ADAS) to collect occupancy information, and broadcast it through a connected vehicle network. The challenge that the data transmitted through ADAS unit has …
Scalable Radar-based Roadside Perception: Self-localization and Occupancy Heat Map for Traffic Analysis
Fraunhofer IVI（弗劳恩霍夫应用信息学研究所（IVI））；Karlsruhe Institute of Technology (KIT)（卡尔斯鲁厄理工学院）；Technische Hochschule Ingolstadt（英戈尔施塔特应用技术大学）；Technical University of Munich（慕尼黑工业大学）；Fraunhofer IOSB（弗劳恩霍夫光学、电子和图像利用研究所（IOSB））
Johns Hopkins University Applied Physics Lab（约翰斯·霍普金斯大学应用物理实验室）；Johns Hopkins University（约翰斯·霍普金斯大学）
Fast, collision-free motion through unknown environments remains a challenging problem for robotic systems. In these situations, the robot's ability to reason about its future motion is often severely limited by sensor field of view (FOV). By contrast, biological systems routinely make decisions by taking into consideration what might exist beyond their FOV based on prior experience. In this paper, we present an approach for predicting occupancy map representations of sensor data for future robot motions using deep neural networks. We evaluate several deep network architectures, including pure…
Traditional dense volumetric representations for robotic mapping make simplifying assumptions about sensor noise characteristics due to computational constraints. We present a framework that, unlike conventional occupancy grid maps, explicitly models the sensor ray formation for a depth sensor via a Markov Random Field and performs loopy belief propagation to infer the marginal probability of occupancy at each voxel in a map. By explicitly reasoning about occlusions our approach models the correlations between adjacent voxels in the map. Further, by incorporating learnt sensor noise characteri…
Institute for Experiential Robotics（体验机器人学研究所）；Northeastern University（东北大学）；Khoury College of Computer Sciences（Khoury计算机科学学院）
This paper presents a Deep Reinforcement Learning based navigation approach in which we define the occupancy observations as heuristic evaluations of motion primitives, rather than using raw sensor data. Our method enables fast mapping of the occupancy data, generated by multi-sensor fusion, into trajectory values in 3D workspace. The computationally efficient trajectory evaluation allows dense sampling of the action space. We utilize our occupancy observations in different data structures to analyze their effects on both training process and navigation performance. We train and test our metho…
FZI Research Center for Information Technology（FZI信息技术研究中心）；University of Vigo（维戈大学）
A detailed environment perception is a crucial component of automated vehicles. However, to deal with the amount of perceived information, we also require segmentation strategies. Based on a grid map environment representation, well-suited for sensor fusion, free-space estimation and machine learning, we detect and classify objects using deep convolutional neural networks. As input for our networks we use a multi-layer grid map efficiently encoding 3D range sensor information. The inference output consists of a list of rotated bounding boxes with associated semantic classes. We conduct extensi…
In perception tasks of automated vehicles (AVs) data-driven have often outperformed conventional approaches. This motivated us to develop a data-driven methodology to compute occupancy grid maps (OGMs) from lidar measurements. Our approach extends previous work such that the estimated environment representation now contains an additional layer for cells occupied by dynamic objects. Earlier solutions could only distinguish between free and occupied cells. The information whether an obstacle could move plays an important role for planning the behavior of an AV. We present two approaches to gener…
Jet Propulsion Laboratory (JPL), California Institute of Technology（喷气推进实验室（JPL），加州理工学院）
Occupancy grids are the most common framework when it comes to creating a map of the environment using a robot. This paper studies occupancy grids from the motion planning perspective and proposes a mapping method that provides richer data (map) for the purpose of planning and collision avoidance. Typically, in occupancy grid mapping, each cell contains a single number representing the probability of cell being occupied. This leads to conflicts in the map, and more importantly inconsistency between the map error and reported confidence values. Such inconsistencies pose challenges for the plann…
The problem of active mapping aims to plan an informative sequence of sensing views given a limited budget such as distance traveled. This paper consider active occupancy grid mapping using a range sensor, such as LiDAR or depth camera. State-of-the-art methods optimize information-theoretic measures relating the occupancy grid probabilities with the range sensor measurements. The non-smooth nature of ray-tracing within a grid representation makes the objective function non-differentiable, forcing existing methods to search over a discrete space of candidate trajectories. This work proposes a …
Traditional approaches to mapping of environments in robotics make use of spatially discretized representations, such as occupancy grid maps. Modern systems, e.g. in agriculture or automotive applications, are equipped with a variety of different sensors to gather diverse process-relevant modalities from the environment. The amount of data and its associated semantic information demand for broader data structures and frameworks, like semantical occupancy grid maps (SOGMs). This multi-modal representation also calls for novel methods of path planning. Due to the sequential nature of path planni…
GRASP Lab, University of Pennsylvania（宾夕法尼亚大学GRASP实验室）
Constructing an occupancy representation of the environment is a fundamental problem for robot autonomy. Many accurate and efficient methods exist that address this problem but most assume that the occupancy states of different elements in the map representation are statistically independent. The focus of this paper is to provide a model that captures correlation of the occupancy of map elements. Correlation is important not only for improved accuracy but also for quantifying uncertainty in the map and for planning autonomous mapping trajectories based on the correlation among known and unknow…
