We present a novel method for testing the safety of self-driving vehicles in simulation. We propose an alternative to sensor simulation, as sensor simulation is expensive and has large domain gaps. Instead, we directly simulate the outputs of the self-driving vehicle's perception and prediction system, enabling realistic motion planning testing. Specifically, we use paired data in the form of ground truth labels and real perception and prediction outputs to train a model that predicts what the online system will produce. Importantly, the inputs to our system consists of high definition maps, bounding boxes, and trajectories, which can be easily sketched by a test engineer in a matter of minutes. This makes our approach a much more scalable solution. Quantitative results on two large-scale datasets demonstrate that we can realistically test motion planning using our simulations.
University of Waterloo（滑铁卢大学）；DENSO CORPORATION（电装公司）
Safety assurance is a central concern for the development and societal acceptance of automated driving (AD) systems. Perception is a key aspect of AD that relies heavily on Machine Learning (ML). Despite the known challenges with the safety assurance of ML-based components, proposals have recently emerged for unit-level safety cases addressing these components. Unfortunately, AD safety cases express safety requirements at the system level and these efforts are missing the critical linking argument needed to integrate safety requirements at the system level with component performance requiremen…
RWTH Aachen University（亚琛工业大学）；Institute for Automotive Engineering（汽车工程研究所）
Safety assurance of automated driving systems must consider uncertain environment perception. This paper reviews literature addressing how perception testing is realized as part of safety assurance. We focus on testing for verification and validation purposes at the interface between perception and planning, and structure our analysis along the three axes 1) test criteria and metrics, 2) test scenarios, and 3) reference data. Furthermore, the analyzed literature includes related safety standards, safety-independent perception algorithm benchmarking, and sensor modeling. We find that the realiz…
Understanding human driving behavior is crucial to develop autonomous vehicles' algorithms. However, most low level automation, such as the one in advanced driving assistance systems (ADAS), is based on objective safety measures, which are not always aligned with what the drivers perceive as safe and their correspondent driving behavior. Finding the bridge between the subjective perception and objective safety measures has been analyzed in this paper focusing specifically on lane-change scenarios. Results showed statistically significant differences between what is perceived as safe by drivers…
Progressive Bird's Eye View Perception for Safety-Critical Autonomous Driving: A Comprehensive Survey
State Key Laboratory of Robotics and System（机器人系统国家重点实验室）；Harbin Institute of Technology（哈尔滨工业大学）；State Key Laboratory of Intelligent Green Vehicle and Mobility（智能绿色车辆与移动性国家重点实验室）；Tsinghua University（清华大学）；the School of Mechanical and Aerospace Engineering（机械与航空航天工程学院）；Nanyang Technological University（南洋理工大学）；Beijing Key Laboratory of Traffic Data Mining and Embodied Intelligence（北京交通数据挖掘与具身智能重点实验室）；Beijing Jiaotong University（北京交通大学）；the Institute for Infocomm Research（信息通信研究所）；A*STAR；the Engineering Cluster（工程集群）；the Singapore Institute of Technology（新加坡理工学院）
The processing requirement of autonomous vehicles (AVs) for high-accuracy perception in complex scenarios can exceed the resources offered by the in-vehicle computer, degrading safety and comfort. This paper proposes a sensor frame processing rate (FPR) estimation model, Zhuyi, that quantifies the minimum safe FPR continuously in a driving scenario. Zhuyi can be employed post-deployment as an online safety check and to prioritize work. Experiments conducted using a multi-camera state-of-the-art industry AV system show that Zhuyi's estimated FPRs are conservative, yet the system can maintain sa…
Safety-Oriented Trajectory Ranking from BEV Perception Outputs
University of the Bundeswehr Munich（德国联邦国防军慕尼黑大学）；Ludwig-Maximilians-Universität München（慕尼黑大学）；Research Institute CODE, University of the Bundeswehr Munich（德国联邦国防军慕尼黑大学CODE研究院）
Agriculture has a high number of fatalities compared to other blue collar fields, additionally population decreasing in rural areas is resulting in decreased work force. These issues have resulted in increased focus on improving efficiency of and introducing autonomy in agriculture. Field robots are an increasingly promising branch of robotics targeted at full automation in agriculture. The safety aspect however is rely addressed in connection with safety standards, which limits the real-world applicability. In this paper we present an analysis of a vision pipeline in connection with functiona…
Learning-based perception and prediction modules in modern autonomous driving systems typically rely on expensive human annotation and are designed to perceive only a handful of predefined object categories. This closed-set paradigm is insufficient for the safety-critical autonomous driving task, where the autonomous vehicle needs to process arbitrarily many types of traffic participants and their motion behaviors in a highly dynamic world. To address this difficulty, this paper pioneers a novel and challenging direction, i.e., training perception and prediction models to understand open-set m…
Uber Advanced Technologies Group（优步先进技术集团）；University of Toronto（多伦多大学）；MIT（麻省理工学院）
Sensor simulation is a key component for testing the performance of self-driving vehicles and for data augmentation to better train perception systems. Typical approaches rely on artists to create both 3D assets and their animations to generate a new scenario. This, however, does not scale. In contrast, we propose to recover the shape and motion of pedestrians from sensor readings captured in the wild by a self-driving car driving around. Towards this goal, we formulate the problem as energy minimization in a deep structured model that exploits human shape priors, reprojection consistency with…
While the most visible part of the safety verification process of automated vehicles concerns the planning and control system, it is often overlooked that safety of the latter crucially depends on the fault-tolerance of the preceding environment perception. Modern perception systems feature complex and often machine-learning-based components with various failure modes that can jeopardize the overall safety. At the same time, a verification by for example redundant execution is not always feasible due to resource constraints. In this paper, we address the need for feasible and efficient percept…
Near-Field Perception for Safety Enhancement of Autonomous Mobile Robots in Manufacturing Environments
Department of Mechanical Engineering, University of Michigan, MI 48109, USA；Materials \& Manufacturing Systems Research, General Motors R \& D, Warren, MI 48092, USA
Injecting Hallucinations in Autonomous Vehicles: A Component-Agnostic Safety Evaluation Framework
Department of Communication, University of Miami, Coral Gables, FL 33124 USA（迈阿密大学通信系）；Department of Communication, Stanford University, Stanford, CA 94305-2050 USA（斯坦福大学通信系）
Autonomous driving technology has drawn a lot of attention due to its fast development and extremely high commercial values. The recent technological leap of autonomous driving can be primarily attributed to the progress in the environment perception. Good environment perception provides accurate high-level environment information which is essential for autonomous vehicles to make safe and precise driving decisions and strategies. Moreover, such progress in accurate environment perception would not be possible without deep learning models and advanced onboard sensors, such as optical sensors (…
Perception Imitation: Towards Synthesis-free Simulator for Autonomous Vehicles
The Chinese University of Hong Kong（香港中文大学）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；Tongji University（同济大学）；Centre for Perceptual and Interactive Intelligence Limited（感知与交互智能中心有限公司）
Huawei Technologies Canada, Ltd.（加拿大华为技术有限公司）；University of Alberta（阿尔伯塔大学）
We propose and demonstrate a framework called perception as prediction for autonomous driving that uses general value functions (GVFs) to learn predictions. Perception as prediction learns data-driven predictions relating to the impact of actions on the agent's perception of the world. It also provides a data-driven approach to predict the impact of the anticipated behavior of other agents on the world without explicitly learning their policy or intentions. We demonstrate perception as prediction by learning to predict an agent's front safety and rear safety with GVFs, which encapsulate antici…
This review paper focuses on different aspects of perceived safety for a number of autonomous physical systems. This is a major aspect of robotics research, as more and more applications allow human and autonomous systems to share their space, with crucial implications both on safety and on its perception. The alternative terms used to express related concepts (e.g., psychological safety, trust, comfort, stress, fear, and anxiety) are listed and explained. Then, the available methods to assess perceived safety (i.e., questionnaires, physiological measurements, behavioral assessment, and direct…
The advance towards higher levels of automation within the field of automated driving is accompanied by increasing requirements for the operational safety of vehicles. Induced by the limitation of computational resources, trade-offs between the computational complexity of algorithms and their potential to ensure safe operation of automated vehicles are often encountered. Situation-aware environment perception presents one promising example, where computational resources are distributed to regions within the perception area that are relevant for the task of the automated vehicle. While prior ma…
PEM: Perception Error Model for Virtual Testing of Autonomous Vehicles
Nanyang Technological University（南洋理工大学）；Agency for Science, Technology and Research (A*STAR)（新加坡科技研究局）；TU Delft（代尔夫特理工大学）；The Hong Kong Polytechnic University（香港理工大学）
Towards Benchmarking and Assessing the Safety and Robustness of Autonomous Driving on Safety-critical Scenarios
Zhongguancun Laboratory（中关村实验室）；Beihang University（北京航空航天大学）；Institute of Data Space, Hefei Comprehensive National Science Center（合肥综合性国家科学中心数据空间研究院）；Institute of Information Science, Beijing Jiaotong University（北京交通大学信息科学研究所）；A*STAR（新加坡科技研究局）
Universidad Nacional de Colombia - UnSecureLab Research Group（哥伦比亚国立大学 - UnSecureLab研究组）；Universidad Nacional de Colombia（哥伦比亚国立大学）
Safety perception measurement has been a subject of interest in many cities of the world. This is due to its social relevance, and to its effect on some local economic activities. Even though people safety perception is a subjective topic, sometimes it is possible to find out common patterns given a restricted geographical and sociocultural context. This paper presents an approach that makes use of image processing and machine learning techniques to detect with high accuracy urban environment patterns that could affect citizen's safety perception.
Deep learning methods are widely regarded as indispensable when it comes to designing perception pipelines for autonomous agents such as robots, drones or automated vehicles. The main reasons, however, for deep learning not being used for autonomous agents at large scale already are safety concerns. Deep learning approaches typically exhibit a black-box behavior which makes it hard for them to be evaluated with respect to safety-critical aspects. While there have been some work on safety in deep learning, most papers typically focus on high-level safety concerns. In this work, we seek to dive …
Valeo Schalter und Sensoren GmbH（法雷奥开关与传感器有限公司）；Volkswagen AG（大众汽车集团）
Safety is one of the most important development goals for highly automated driving (HAD) systems. This applies in particular to the perception function driven by Deep Neural Networks (DNNs). For these, large parts of the traditional safety processes and requirements are not fully applicable or sufficient. The aim of this paper is to present a framework for the description and mitigation of DNN insufficiencies and the derivation of relevant safety mechanisms to increase the safety of DNNs. To assess the effectiveness of these safety mechanisms, we present a categorization scheme for evaluation …
