The processing requirement of autonomous vehicles (AVs) for high-accuracy perception in complex scenarios can exceed the resources offered by the in-vehicle computer, degrading safety and comfort. This paper proposes a sensor frame processing rate (FPR) estimation model, Zhuyi, that quantifies the minimum safe FPR continuously in a driving scenario. Zhuyi can be employed post-deployment as an online safety check and to prioritize work. Experiments conducted using a multi-camera state-of-the-art industry AV system show that Zhuyi's estimated FPRs are conservative, yet the system can maintain safety by processing only 36% or fewer frames compared to a default 30-FPR system in the tested scenarios.
University of Waterloo（滑铁卢大学）；DENSO CORPORATION（电装公司）
Safety assurance is a central concern for the development and societal acceptance of automated driving (AD) systems. Perception is a key aspect of AD that relies heavily on Machine Learning (ML). Despite the known challenges with the safety assurance of ML-based components, proposals have recently emerged for unit-level safety cases addressing these components. Unfortunately, AD safety cases express safety requirements at the system level and these efforts are missing the critical linking argument needed to integrate safety requirements at the system level with component performance requiremen…
Understanding human driving behavior is crucial to develop autonomous vehicles' algorithms. However, most low level automation, such as the one in advanced driving assistance systems (ADAS), is based on objective safety measures, which are not always aligned with what the drivers perceive as safe and their correspondent driving behavior. Finding the bridge between the subjective perception and objective safety measures has been analyzed in this paper focusing specifically on lane-change scenarios. Results showed statistically significant differences between what is perceived as safe by drivers…
RWTH Aachen University（亚琛工业大学）；Institute for Automotive Engineering（汽车工程研究所）
Safety assurance of automated driving systems must consider uncertain environment perception. This paper reviews literature addressing how perception testing is realized as part of safety assurance. We focus on testing for verification and validation purposes at the interface between perception and planning, and structure our analysis along the three axes 1) test criteria and metrics, 2) test scenarios, and 3) reference data. Furthermore, the analyzed literature includes related safety standards, safety-independent perception algorithm benchmarking, and sensor modeling. We find that the realiz…
Agriculture has a high number of fatalities compared to other blue collar fields, additionally population decreasing in rural areas is resulting in decreased work force. These issues have resulted in increased focus on improving efficiency of and introducing autonomy in agriculture. Field robots are an increasingly promising branch of robotics targeted at full automation in agriculture. The safety aspect however is rely addressed in connection with safety standards, which limits the real-world applicability. In this paper we present an analysis of a vision pipeline in connection with functiona…
PEM: Perception Error Model for Virtual Testing of Autonomous Vehicles
Nanyang Technological University（南洋理工大学）；Agency for Science, Technology and Research (A*STAR)（新加坡科技研究局）；TU Delft（代尔夫特理工大学）；The Hong Kong Polytechnic University（香港理工大学）
Uber Advanced Technologies Group（优步先进技术集团）；University of Toronto（多伦多大学）；Shanghai Jiao Tong University（上海交通大学）
We present a novel method for testing the safety of self-driving vehicles in simulation. We propose an alternative to sensor simulation, as sensor simulation is expensive and has large domain gaps. Instead, we directly simulate the outputs of the self-driving vehicle's perception and prediction system, enabling realistic motion planning testing. Specifically, we use paired data in the form of ground truth labels and real perception and prediction outputs to train a model that predicts what the online system will produce. Importantly, the inputs to our system consists of high definition maps, b…
Towards Benchmarking and Assessing the Safety and Robustness of Autonomous Driving on Safety-critical Scenarios
Zhongguancun Laboratory（中关村实验室）；Beihang University（北京航空航天大学）；Institute of Data Space, Hefei Comprehensive National Science Center（合肥综合性国家科学中心数据空间研究院）；Institute of Information Science, Beijing Jiaotong University（北京交通大学信息科学研究所）；A*STAR（新加坡科技研究局）
Huawei Technologies Canada, Ltd.（加拿大华为技术有限公司）；University of Alberta（阿尔伯塔大学）
We propose and demonstrate a framework called perception as prediction for autonomous driving that uses general value functions (GVFs) to learn predictions. Perception as prediction learns data-driven predictions relating to the impact of actions on the agent's perception of the world. It also provides a data-driven approach to predict the impact of the anticipated behavior of other agents on the world without explicitly learning their policy or intentions. We demonstrate perception as prediction by learning to predict an agent's front safety and rear safety with GVFs, which encapsulate antici…
Progressive Bird's Eye View Perception for Safety-Critical Autonomous Driving: A Comprehensive Survey
State Key Laboratory of Robotics and System（机器人系统国家重点实验室）；Harbin Institute of Technology（哈尔滨工业大学）；State Key Laboratory of Intelligent Green Vehicle and Mobility（智能绿色车辆与移动性国家重点实验室）；Tsinghua University（清华大学）；the School of Mechanical and Aerospace Engineering（机械与航空航天工程学院）；Nanyang Technological University（南洋理工大学）；Beijing Key Laboratory of Traffic Data Mining and Embodied Intelligence（北京交通数据挖掘与具身智能重点实验室）；Beijing Jiaotong University（北京交通大学）；the Institute for Infocomm Research（信息通信研究所）；A*STAR；the Engineering Cluster（工程集群）；the Singapore Institute of Technology（新加坡理工学院）
While the most visible part of the safety verification process of automated vehicles concerns the planning and control system, it is often overlooked that safety of the latter crucially depends on the fault-tolerance of the preceding environment perception. Modern perception systems feature complex and often machine-learning-based components with various failure modes that can jeopardize the overall safety. At the same time, a verification by for example redundant execution is not always feasible due to resource constraints. In this paper, we address the need for feasible and efficient percept…
Near-Field Perception for Safety Enhancement of Autonomous Mobile Robots in Manufacturing Environments
Department of Mechanical Engineering, University of Michigan, MI 48109, USA；Materials \& Manufacturing Systems Research, General Motors R \& D, Warren, MI 48092, USA
Injecting Hallucinations in Autonomous Vehicles: A Component-Agnostic Safety Evaluation Framework
Department of Communication, University of Miami, Coral Gables, FL 33124 USA（迈阿密大学通信系）；Department of Communication, Stanford University, Stanford, CA 94305-2050 USA（斯坦福大学通信系）
Safety-Oriented Trajectory Ranking from BEV Perception Outputs
University of the Bundeswehr Munich（德国联邦国防军慕尼黑大学）；Ludwig-Maximilians-Universität München（慕尼黑大学）；Research Institute CODE, University of the Bundeswehr Munich（德国联邦国防军慕尼黑大学CODE研究院）
This review paper focuses on different aspects of perceived safety for a number of autonomous physical systems. This is a major aspect of robotics research, as more and more applications allow human and autonomous systems to share their space, with crucial implications both on safety and on its perception. The alternative terms used to express related concepts (e.g., psychological safety, trust, comfort, stress, fear, and anxiety) are listed and explained. Then, the available methods to assess perceived safety (i.e., questionnaires, physiological measurements, behavioral assessment, and direct…
The advance towards higher levels of automation within the field of automated driving is accompanied by increasing requirements for the operational safety of vehicles. Induced by the limitation of computational resources, trade-offs between the computational complexity of algorithms and their potential to ensure safe operation of automated vehicles are often encountered. Situation-aware environment perception presents one promising example, where computational resources are distributed to regions within the perception area that are relevant for the task of the automated vehicle. While prior ma…
Learning-based perception and prediction modules in modern autonomous driving systems typically rely on expensive human annotation and are designed to perceive only a handful of predefined object categories. This closed-set paradigm is insufficient for the safety-critical autonomous driving task, where the autonomous vehicle needs to process arbitrarily many types of traffic participants and their motion behaviors in a highly dynamic world. To address this difficulty, this paper pioneers a novel and challenging direction, i.e., training perception and prediction models to understand open-set m…
Stanford University（斯坦福大学）；University of Edinburgh（爱丁堡大学）
Modern autonomous systems rely on perception modules to process complex sensor measurements into state estimates. These estimates are then passed to a controller, which uses them to make safety-critical decisions. It is therefore important that we design perception systems to minimize errors that reduce the overall safety of the system. We develop a risk-driven approach to designing perception systems that accounts for the effect of perceptual errors on the performance of the fully-integrated, closed-loop system. We formulate a risk function to quantify the effect of a given perceptual error o…
Although artificial intelligence-based perception (AIP) using deep neural networks (DNN) has achieved near human level performance, its well-known limitations are obstacles to the safety assurance needed in autonomous applications. These include vulnerability to adversarial inputs, inability to handle novel inputs and non-interpretability. While research in addressing these limitations is active, in this paper, we argue that a fundamentally different approach is needed to address them. Inspired by dual process models of human cognition, where Type 1 thinking is fast and non-conscious while Typ…
Data-driven models (DDM) based on machine learning and other AI techniques play an important role in the perception of increasingly autonomous systems. Due to the merely implicit definition of their behavior mainly based on the data used for training, DDM outputs are subject to uncertainty. This poses a challenge with respect to the realization of safety-critical perception tasks by means of DDMs. A promising approach to tackling this challenge is to estimate the uncertainty in the current situation during operation and adapt the system behavior accordingly. In previous work, we focused on run…
Impact Analysis of Inference Time Attack of Perception Sensors on Autonomous Vehicles
Oak Ridge National Laboratory（橡树岭国家实验室）；The University of Texas at Dallas（德克萨斯大学达拉斯分校）；The Hong Kong University of Science and Technology（香港科技大学）；Purdue University（普渡大学）
AccidentGPT: Accident Analysis and Prevention from V2X Environmental Perception with Multi-modal Large Model
State Key Lab of Intelligent Transportation System, Beihang University（北京航空航天大学智能交通系统国家重点实验室）；Zhongguancun Laboratory（中关村实验室）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；The University of Hong Kong（香港大学）
