Today's mobile robots are expected to operate in complex environments they share with humans. To allow intuitive human-robot collaboration, robots require a human-like understanding of their surroundings in terms of semantically classified instances. In this paper, we propose a new approach for deriving such instance-based semantic maps purely from occupancy grids. We employ a combination of deep learning techniques to detect, segment and extract door hypotheses from a random-sized map. The extraction is followed by a post-processing chain to further increase the accuracy of our approach, as well as place categorization for the three classes room, door and corridor. All detected and classified entities are described as instances specified in a common coordinate system, while a topological map is derived to capture their spatial links. To train our two neural networks used for detection and map segmentation, we contribute a simulator that automatically creates and annotates the required training data. We further provide insight into which features are learned to detect doorways, and how the simulated training data can be augmented to train networks for the direct application on real-world grid maps. We evaluate our approach on several publicly available real-world data sets. Even though the used networks are solely trained on simulated data, our approach demonstrates high robustness and effectiveness in various real-world indoor environments.
Wuyi University（五邑大学）；The Chinese University of Hong Kong（香港中文大学）；irobotix Co.,Ltd（3irobotix有限公司）；Macau University of Science and Technology（澳门科技大学）
The occupancy grid map is a critical component of autonomous positioning and navigation in the mobile robotic system, as many other systems' performance depends heavily on it. To guarantee the quality of the occupancy grid maps, researchers previously had to perform tedious manual recognition for a long time. This work focuses on automatic abnormal occupancy grid map recognition using the residual neural networks and a novel attention mechanism module. We propose an effective channel and spatial Residual SE(csRSE) attention module, which contains a residual block for producing hierarchical fea…
Prediction of dynamic environment is crucial to safe navigation of an autonomous vehicle. Urban traffic scenes are particularly challenging to forecast due to complex interactions between various dynamic agents, such as vehicles and vulnerable road users. Previous approaches have used egocentric occupancy grid maps to represent and predict dynamic environments. However, these predictions suffer from blurriness, loss of scene structure at turns, and vanishing of agents over longer prediction horizon. In this work, we propose a novel framework to make long-term predictions by representing the tr…
TU Dortmund（多特蒙德工业大学）；HELLA Aglaia Mobile Vision GmbH（海拉阿格莱亚移动视觉有限公司）
Environment modeling utilizing sensor data fusion and object tracking is crucial for safe automated driving. In recent years, the classical occupancy grid map approach, which assumes a static environment, has been extended to dynamic occupancy grid maps, which maintain the possibility of a low-level data fusion while also estimating the position and velocity distribution of the dynamic local environment. This paper presents the further development of a previous approach. To the best of the author's knowledge, there is no publication about dynamic occupancy grid mapping with subsequent analysis…
University of Grenoble Alpes（格勒诺布尔阿尔卑斯大学）；Inria（法国国家信息与自动化研究所）；Univ Lyon（里昂大学）；INSA Lyon（里昂国立应用科学学院）
Reliably predicting future occupancy of highly dynamic urban environments is an important precursor for safe autonomous navigation. Common challenges in the prediction include forecasting the relative position of other vehicles, modelling the dynamics of vehicles subjected to different traffic conditions, and vanishing surrounding objects. To tackle these challenges, we propose a spatio-temporal prediction network pipeline that takes the past information from the environment and semantic labels separately for generating future occupancy predictions. Compared to the current SOTA, our approach p…
Renault S.A.S（雷诺股份有限公司）；Sorbonne universités, Université de technologie de Compiègne, CNRS, Heudiasyc（索邦大学，贡比涅技术大学，法国国家科学研究中心，Heudiasyc实验室）；Monash University（莫纳什大学）
Camera-based end-to-end driving neural networks bring the promise of a low-cost system that maps camera images to driving control commands. These networks are appealing because they replace laborious hand engineered building blocks but their black-box nature makes them difficult to delve in case of failure. Recent works have shown the importance of using an explicit intermediate representation that has the benefits of increasing both the interpretability and the accuracy of networks' decisions. Nonetheless, these camera-based networks reason in camera view where scale is not homogeneous and he…
Karlsruhe Institute of Technology (KIT)（卡尔斯鲁厄理工学院）；FZI Research Center for Information Technology（FZI信息技术研究中心）
We present a generic evidential grid mapping pipeline designed for imaging sensors such as LiDARs and cameras. Our grid-based evidential model contains semantic estimates for cell occupancy and ground separately. We specify the estimation steps for input data represented by point sets, but mainly focus on input data represented by images such as disparity maps or LiDAR range images. Instead of relying on an external ground segmentation only, we deduce occupancy evidence by analyzing the surface orientation around measurements. We conduct experiments and evaluate the presented method using LiDA…
The University of Hong Kong（香港大学）；Fuzhou University（福州大学）
In this paper, we aim to forecast a future trajectory distribution of a moving agent in the real world, given the social scene images and historical trajectories. Yet, it is a challenging task because the ground-truth distribution is unknown and unobservable, while only one of its samples can be applied for supervising model learning, which is prone to bias. Most recent works focus on predicting diverse trajectories in order to cover all modes of the real distribution, but they may despise the precision and thus give too much credit to unrealistic predictions. To address the issue, we learn th…
Occupancy grid maps (OGMs) are fundamental to most systems for autonomous robotic navigation. However, CPU-based implementations struggle to keep up with data rates from modern 3D lidar sensors, and provide little capacity for modern extensions which maintain richer voxel representations. This paper presents OHM, our open source, GPU-based OGM framework. We show how the algorithms can be mapped to GPU resources, resolving difficulties with contention to obtain a successful implementation. The implementation supports many modern OGM algorithms including NDT-OM, NDT-TM, decay-rate and TSDF. A th…
Deep Height Decoupling for Precise Vision-based 3D Occupancy Prediction
School of Computer Science and Engineering, Nanjing University of Science and Technology（南京理工大学计算机科学与工程学院）；College of Computer Science, Nankai University（南开大学计算机学院）；Electronics and Information, Northwestern Polytechnical University（西北工业大学电子信息学院）
Fully Convolutional Neural Networks for Dynamic Object Detection in Grid Maps (Masters Thesis)
FZI Research Center for Information Technology（FZI信息技术研究中心）；KIT – University of the State of Baden-Wuerttemberg and National Research Center of the Helmholtz Association（卡尔斯鲁厄理工学院 – 巴登-符腾堡州大学及亥姆霍兹联合会国家研究中心）
RenderOcc: Vision-Centric 3D Occupancy Prediction with 2D Rendering Supervision
National Key Laboratory for Multimedia Information Processing（多媒体信息处理国家重点实验室）；School of CS, Peking University（北京大学计算机学院）；Xiaomi Car（小米汽车）；Nanjing University（南京大学）；Nanyang Technological University（南洋理工大学）；CUHK MMLAB（香港中文大学多媒体实验室）
University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-尚佩恩分校）
Single-view RGB-D human reconstruction with implicit functions is often formulated as per-point classification. Specifically, a set of 3D locations within the view-frustum of the camera are first projected independently onto the image and a corresponding feature is subsequently extracted for each 3D location. The feature of each 3D location is then used to classify independently whether the corresponding 3D point is inside or outside the observed object. This procedure leads to sub-optimal results because correlations between predictions for neighboring locations are only taken into account im…
