This paper presents two variations of a novel stochastic prediction algorithm that enables mobile robots to accurately and robustly predict the future state of complex dynamic scenes. The proposed algorithm uses a variational autoencoder to predict a range of possible future states of the environment. The algorithm takes full advantage of the motion of the robot itself, the motion of dynamic objects, and the geometry of static objects in the scene to improve prediction accuracy. Three simulated and real-world datasets collected by different robot models are used to demonstrate that the proposed algorithm is able to achieve more accurate and robust prediction performance than other prediction algorithms. Furthermore, a predictive uncertainty-aware planner is proposed to demonstrate the effectiveness of the proposed predictor in simulation and real-world navigation experiments. Implementations are open source at https://github.com/TempleRAIL/SOGMP.
This paper describes a system whereby a robot detects and track human-meaningful navigational cues as it navigates in an indoor environment. It is intended as the sensor front-end for a mobile robot system that can communicate its navigational context with human users. From simulated LiDAR scan data we construct a set of 2D occupancy grid bitmaps, then hand-label these with human-scale navigational features such as closed doors, open corridors and intersections. We train a Convolutional Neural Network (CNN) to recognize these features on input bitmaps. In our demonstration system, these featur…
Ulm University（乌尔姆大学）；Robert Bosch GmbH（罗伯特·博世有限公司）；Otto von Guericke University Magdeburg（马格德堡奥托·冯·格里克大学）
A common approach for modeling the environment of an autonomous vehicle are dynamic occupancy grid maps, in which the surrounding is divided into cells, each containing the occupancy and velocity state of its location. Despite the advantage of modeling arbitrary shaped objects, the used algorithms rely on hand-designed inverse sensor models and semantic information is missing. Therefore, we introduce a multi-task recurrent neural network to predict grid maps providing occupancies, velocity estimates, semantic information and the driveable area. During training, our network architecture, which …
The comprehensive representation and understanding of the driving environment is crucial to improve the safety and reliability of autonomous vehicles. In this paper, we present a new approach to establish an environment model containing a segmentation between static and dynamic background and parametric modeled objects with shape, position and orientation. Multiple laser scanners are fused into a dynamic occupancy grid map resulting in a 360° perception of the environment. A single-stage deep convolutional neural network is combined with a recurrent neural network, which takes a time series of…
Formerly I presented a metric navigation method in the Webots mobile robot simulator. The navigating Khepera-like robot builds an occupancy grid of the environment and explores the square-shaped room around with a value iteration algorithm. Now I created a topological navigation procedure based on the occupancy grid process. The extension by a skeletonization algorithm results a graph of important places and the connecting routes among them. I also show the significant time profit gained during the process.
Università degli studi di Milano（米兰大学）；Örebro Universitet（厄勒布鲁大学）；Aalto University（阿尔托大学）；Politecnico di Milano（米兰理工大学）
Identifying the environment's structure, i.e., to detect core components as rooms and walls, can facilitate several tasks fundamental for the successful operation of indoor autonomous mobile robots, including semantic environment understanding. These robots often rely on 2D occupancy maps for core tasks such as localisation and motion and task planning. However, reliable identification of structure and room segmentation from 2D occupancy maps is still an open problem due to clutter (e.g., furniture and movable object), occlusions, and partial coverage. We propose a method for the RObust Struct…
In the context of autonomous vehicles, one of the most crucial tasks is to estimate the risk of the undertaken action. While navigating in complex urban environments, the Bayesian occupancy grid is one of the most popular types of maps, where the information of occupancy is stored as the probability of collision. Although widely used, this kind of representation is not well suited for risk assessment: because of its discrete nature, the probability of collision becomes dependent on the tessellation size. Therefore, risk assessments on Bayesian occupancy grids cannot yield risks with meaningful…
Drivable free space information is vital for autonomous vehicles that have to plan evasive maneuvers in real-time. In this paper, we present a new efficient method for environmental free space detection with laser scanner based on 2D occupancy grid maps (OGM) to be used for Advanced Driving Assistance Systems (ADAS) and Collision Avoidance Systems (CAS). Firstly, we introduce an enhanced inverse sensor model tailored for high-resolution laser scanners for building OGM. It compensates the unreflected beams and deals with the ray casting to grid cells accuracy and computational effort problems. …
Modeling and understanding the environment is an essential task for autonomous driving. In addition to the detection of objects, in complex traffic scenarios the motion of other road participants is of special interest. Therefore, we propose to use a recurrent neural network to predict a dynamic occupancy grid map, which divides the vehicle surrounding in cells, each containing the occupancy probability and a velocity estimate. During training, our network is fed with sequences of measurement grid maps, which encode the lidar measurements of a single time step. Due to the combination of convol…
Environment modeling in autonomous driving is realized by two fundamental approaches, grid-based and feature-based approach. Both methods interpret the environment differently and show some situation-dependent beneficial realizations. In order to use the advantages of both methods, a combination makes sense. This work presents a fusion, which establishes an association between the representations of environment modeling and then decoupled from this performs a fusion of the information. Thus, there is no need to adapt the environment models. The developed fusion generates new hypotheses, which …
Institute of Measurement, Control, and Microtechnology, Ulm University（乌尔姆大学测量、控制与微技术研究所）
Long-term situation prediction plays a crucial role in the development of intelligent vehicles. A major challenge still to overcome is the prediction of complex downtown scenarios with multiple road users, e.g., pedestrians, bikes, and motor vehicles, interacting with each other. This contribution tackles this challenge by combining a Bayesian filtering technique for environment representation, and machine learning as long-term predictor. More specifically, a dynamic occupancy grid map is utilized as input to a deep convolutional neural network. This yields the advantage of using spatially dis…
College of Engineering, University of Michigan（密歇根大学工程学院）；University of Technology Sydney（悉尼科技大学）
Most of the existing robotic exploration schemes use occupancy grid representations and geometric targets known as frontiers. The occupancy grid representation relies on the assumption of independence between grid cells and ignores structural correlations present in the environment. We develop a Gaussian Processes (GPs) occupancy mapping technique that is computationally tractable for online map building due to its incremental formulation and provides a continuous model of uncertainty over the map spatial coordinates. The standard way to represent geometric frontiers extracted from occupancy m…
UnsOcc: 3D Semantic Occupancy Prediction in Unstructured Scene via Rendering Fusion
School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；Waytous Inc.（Waytous公司）
The advance towards higher levels of automation within the field of automated driving is accompanied by increasing requirements for the operational safety of vehicles. Induced by the limitation of computational resources, trade-offs between the computational complexity of algorithms and their potential to ensure safe operation of automated vehicles are often encountered. Situation-aware environment perception presents one promising example, where computational resources are distributed to regions within the perception area that are relevant for the task of the automated vehicle. While prior ma…
Prediction of dynamic environment is crucial to safe navigation of an autonomous vehicle. Urban traffic scenes are particularly challenging to forecast due to complex interactions between various dynamic agents, such as vehicles and vulnerable road users. Previous approaches have used egocentric occupancy grid maps to represent and predict dynamic environments. However, these predictions suffer from blurriness, loss of scene structure at turns, and vanishing of agents over longer prediction horizon. In this work, we propose a novel framework to make long-term predictions by representing the tr…
We propose Occupancy Flow Fields, a new representation for motion forecasting of multiple agents, an important task in autonomous driving. Our representation is a spatio-temporal grid with each grid cell containing both the probability of the cell being occupied by any agent, and a two-dimensional flow vector representing the direction and magnitude of the motion in that cell. Our method successfully mitigates shortcomings of the two most commonly-used representations for motion forecasting: trajectory sets and occupancy grids. Although occupancy grids efficiently represent the probabilistic l…
We introduce a motion forecasting (behavior prediction) method that meets the latency requirements for autonomous driving in dense urban environments without sacrificing accuracy. A whole-scene sparse input representation allows StopNet to scale to predicting trajectories for hundreds of road agents with reliable latency. In addition to predicting trajectories, our scene encoder lends itself to predicting whole-scene probabilistic occupancy grids, a complementary output representation suitable for busy urban environments. Occupancy grids allow the AV to reason collectively about the behavior o…
Universidade Federal do Espírito Santo（圣灵联邦大学）；Instituto Federal do Espírito Santo（圣灵联邦研究所）
In this work, we present a novel strategy for correcting imperfections in occupancy grid maps called map decay. The objective of map decay is to correct invalid occupancy probabilities of map cells that are unobservable by sensors. The strategy was inspired by an analogy between the memory architecture believed to exist in the human brain and the maps maintained by an autonomous vehicle. It consists in merging sensory information obtained during runtime (online) with a priori data from a high-precision map constructed offline. In map decay, cells observed by sensors are updated using tradition…
PanoOcc: Unified Occupancy Representation for Camera-based 3D Panoptic Segmentation
Institute of Automation, Chinese Academy of Sciences (CASIA)（中国科学院自动化研究所）；School of Artificial Intelligence, University of Chinese Academy of Sciences (UCAS)（中国科学院大学人工智能学院）；Centre for Artificial Intelligence and Robotics, HKISI, CAS（中国科学院香港创新研究院人工智能与机器人创新中心）
In this work, we tackle the problem of modeling the vehicle environment as dynamic occupancy grid map in complex urban scenarios using recurrent neural networks. Dynamic occupancy grid maps represent the scene in a bird's eye view, where each grid cell contains the occupancy probability and the two dimensional velocity. As input data, our approach relies on measurement grid maps, which contain occupancy probabilities, generated with lidar measurements. Given this configuration, we propose a recurrent neural network architecture to predict a dynamic occupancy grid map, i.e. filtered occupancy a…
This paper proposes a perception and path planning pipeline for autonomous racing in an unknown bounded course. The pipeline was initially created for the 2021 evGrandPrix autonomous division and was further improved for the 2022 event, both of which resulting in first place finishes. Using a simple LiDAR-based perception pipeline feeding into an occupancy grid based expansion algorithm, we determine a goal point to drive. This pipeline successfully achieved reliable and consistent laps in addition with occupancy grid algorithm to know the ways around a cone-defined track with an averaging spe…
Jet Propulsion Laboratory (JPL), California Institute of Technology（喷气推进实验室（JPL），加州理工学院）
Occupancy grids are the most common framework when it comes to creating a map of the environment using a robot. This paper studies occupancy grids from the motion planning perspective and proposes a mapping method that provides richer data (map) for the purpose of planning and collision avoidance. Typically, in occupancy grid mapping, each cell contains a single number representing the probability of cell being occupied. This leads to conflicts in the map, and more importantly inconsistency between the map error and reported confidence values. Such inconsistencies pose challenges for the plann…
Autonomously driving vehicles require a complete and robust perception of the local environment. A main challenge is to perceive any other road users, where multi-object tracking or occupancy grid maps are commonly used. The presented approach combines both methods to compensate false positives and receive a complementary environment perception. Therefore, an environment perception framework is introduced that defines a common representation, extracts objects from a dynamic occupancy grid map and fuses them with tracks of a Labeled Multi-Bernoulli filter. Finally, a confidence value is develop…
FZI Research Center for Information Technology（FZI信息技术研究中心）；Karlsruhe Institute of Technology (KIT)（卡尔斯鲁厄理工学院（KIT））
A detailed environment representation is a crucial component of automated vehicles. Using single range sensor scans, data is often too sparse and subject to occlusions. Therefore, we present a method to augment occupancy grid maps from single views to be similar to evidential occupancy maps acquired from different views using Deep Learning. To accomplish this, we estimate motion between subsequent range sensor measurements and create an evidential 3D voxel map in an extensive post-processing step. Within this voxel map, we explicitly model uncertainty using evidence theory and create a 2D proj…
Mercedes-Benz Research & Development North America, Inc.（梅赛德斯-奔驰北美研发有限公司）；Institute of Measurement, Control, and Microtechnology, Ulm University（乌尔姆大学测量、控制与微技术研究所）；Ulm University（乌尔姆大学）
Grid mapping is a well established approach for environment perception in robotic and automotive applications. Early work suggests estimating the occupancy state of each grid cell in a robot's environment using a Bayesian filter to recursively combine new measurements with the current posterior state estimate of each grid cell. This filter is often referred to as binary Bayes filter (BBF). A basic assumption of classical occupancy grid maps is a stationary environment. Recent publications describe bottom-up approaches using particles to represent the dynamic state of a grid cell and outline pr…
Traditional approaches to mapping of environments in robotics make use of spatially discretized representations, such as occupancy grid maps. Modern systems, e.g. in agriculture or automotive applications, are equipped with a variety of different sensors to gather diverse process-relevant modalities from the environment. The amount of data and its associated semantic information demand for broader data structures and frameworks, like semantical occupancy grid maps (SOGMs). This multi-modal representation also calls for novel methods of path planning. Due to the sequential nature of path planni…
In a context of autonomous robots, one of the most important task is to ensure the safety of the robot and its surrounding. Most of the time, the risk of navigation is simply said to be the probability of collision. This notion of risk is not well defined in the literature, especially when dealing with occupancy grids. The Bayesian occupancy grid is the most used method to deal with complex environments. However, this is not fitted to compute the risk along a path by its discrete nature, hence giving poor results. In this article, we present a new way to store the occupancy of the environment …
IBM T. J. Watson Research Center（IBM T. J. 沃森研究中心）
In this paper we provide an overview of a new framework for robot perception, real-world modelling, and navigation that uses a stochastic tesselated representation of spatial information called the Occupancy Grid. The Occupancy Grid is a multi-dimensional random field model that maintains probabilistic estimates of the occupancy state of each cell in a spatial lattice. Bayesian estimation mechanisms employing stochastic sensor models allow incremental updating of the Occupancy Grid using multi-view, multi-sensor data, composition of multiple maps, decision-making, and incorporation of robot an…
D-BDM: A Direct and Efficient Boundary-Based Occupancy Grid Mapping Framework for LiDARs
Mechatronics and Robotic Systems (MaRS) Laboratory, Department of Mechanical Engineering, The University of Hong Kong（机电系统（MaRS）实验室，机械工程系，香港大学）；Department of Robotics, Perception, and Learning, KTH Royal Institute of Technology（机器人、感知与学习系，皇家理工学院）
University of Toronto Institute for Aerospace Studies (UTIAS)（多伦多大学航空航天研究所）；ETH Zürich（苏黎世联邦理工学院）；Apple Inc.（苹果公司）
We present a novel method for generating, predicting, and using Spatiotemporal Occupancy Grid Maps (SOGM), which embed future information of dynamic scenes. Our automated generation process creates groundtruth SOGMs from previous navigation data. We build on prior work to annotate lidar points based on their dynamic properties, which are then projected on time-stamped 2D grids: SOGMs. We design a 3D-2D feedforward architecture, trained to predict the future time steps of SOGMs, given 3D lidar frames as input. Our pipeline is entirely self-supervised, thus enabling lifelong learning for robots.…
Memory-Efficient Boundary Map for Large-Scale Occupancy Grid Mapping
Mechatronics and Robotic Systems (MaRS) Laboratory, Department of Mechanical Engineering, The University of Hong Kong, Hong Kong SAR, China（机械电子与机器人系统实验室，机械工程系，香港大学，香港特别行政区，中国）；Department of Robotics, Perception, and Learning, KTH Royal Institute of Technology, Stockholm, Sweden（机器人、感知与学习系，皇家理工学院，斯德哥尔摩，瑞典）
Grid-based Submap Joining: An Efficient Algorithm for Simultaneously Optimizing Global Occupancy Map and Local Submap Frames
Faculty of Engineering and Information Technology, University of Technology Sydney（悉尼科技大学工程与信息技术学院）；School of Informatics, University of Edinburgh（爱丁堡大学信息学院）
Cornell University（康奈尔大学）；Massachusetts Institute of Technology（麻省理工学院）
This paper investigates the usefulness of reasoning about the uncertain presence of obstacles during path planning, which typically stems from the usage of probabilistic occupancy grid maps for representing the environment when mapping via a noisy sensor like a stereo camera. The traditional planning paradigm prescribes using a hard threshold on the occupancy probability to declare that a cell is an obstacle, and to plan a single path accordingly while treating unknown space as free. We compare this approach against a new uncertainty-aware planner, which plans two different path hypotheses and…
In perception tasks of automated vehicles (AVs) data-driven have often outperformed conventional approaches. This motivated us to develop a data-driven methodology to compute occupancy grid maps (OGMs) from lidar measurements. Our approach extends previous work such that the estimated environment representation now contains an additional layer for cells occupied by dynamic objects. Earlier solutions could only distinguish between free and occupied cells. The information whether an obstacle could move plays an important role for planning the behavior of an AV. We present two approaches to gener…
