Safety validation is important during the development of safety-critical autonomous systems but can require significant computational effort. Existing algorithms often start from scratch each time the system under test changes. We apply transfer learning to improve the efficiency of reinforcement learning based safety validation algorithms when applied to related systems. Knowledge from previous safety validation tasks is encoded through the action value function and transferred to future tasks with a learned set of attention weights. Including a learned state and action value transformation for each source task can improve performance even when systems have substantially different failure modes. We conduct experiments on safety validation tasks in gridworld and autonomous driving scenarios. We show that transfer learning can improve the initial and final performance of validation algorithms and reduce the number of training steps.
An open problem for autonomous driving is how to validate the safety of an autonomous vehicle in simulation. Automated testing procedures can find failures of an autonomous system but these failures may be difficult to interpret due to their high dimensionality and may be so unlikely as to not be important. This work describes an approach for finding interpretable failures of an autonomous system. The failures are described by signal temporal logic expressions that can be understood by a human, and are optimized to produce failures that have high likelihood. Our methodology is demonstrated for…
Stanford University（斯坦福大学）；NASA Ames Research Center（NASA艾姆斯研究中心）
An open question in autonomous driving is how best to use simulation to validate the safety of autonomous vehicles. Existing techniques rely on simulated rollouts, which can be inefficient for finding rare failure events, while other techniques are designed to only discover a single failure. In this work, we present a new safety validation approach that attempts to estimate the distribution over failures of an autonomous policy using approximate dynamic programming. Knowledge of this distribution allows for the efficient discovery of many failure examples. To address the problem of scalability…
ADDT -- A Digital Twin Framework for Proactive Safety Validation in Autonomous Driving Systems
Shenzhen Institute of Artificial Intelligence and Robotics for Society（深圳市人工智能与机器人研究院）；University of California, Irvine（加州大学欧文分校）；Georgia Institute of Technology（佐治亚理工学院）；South China University of Technology（华南理工大学）
Engineering Psychological Safety in Autonomous Vehicles: A Systems-Theoretic Framework for Psychological Safety in Autonomous Vehicles and its Validation in Real-World Scenarios
Capgemini Engineering（凯捷工程）；University of Oxford（牛津大学）；University of the West of England（西英格兰大学）
SinD 2.0: A Multi-City UAV Dataset with Semantic Risk Annotations for SOTIF-Oriented Safety Validation at Signalized Intersections
Tsinghua University（清华大学）；Beijing Institute of Technology（北京理工大学）；Guangzhou Automobile Group Co., Ltd.（广州汽车集团股份有限公司）；Jilin University（吉林大学）；Chang’an University（长安大学）；Chongqing University（重庆大学）；Southwest University（西南大学）；Wuhan University of Technology（武汉理工大学）；Virginia Tech Transportation Institute（弗吉尼亚理工大学交通研究所）；Virginia Tech（弗吉尼亚理工大学）
How many scenarios are sufficient to validate the safe Operational Design Domain (ODD) of an Automated Driving System (ADS) equipped vehicle? Is a more significant number of sampled scenarios guaranteeing a more accurate safety assessment of the ADS? Despite the various empirical success of ADS safety evaluation with scenario sampling in practice, some of the fundamental properties are largely unknown. This paper seeks to remedy this gap by formulating and tackling the scenario sampling safety assurance problem from a set invariance perspective. First, a novel conceptual equivalence is drawn b…
Ohio State University（俄亥俄州立大学）；Southern University of Science and Technology (SUSTech)（南方科技大学）
The dynamic response of the legged robot locomotion is non-Lipschitz and can be stochastic due to environmental uncertainties. To test, validate, and characterize the safety performance of legged robots, existing solutions on observed and inferred risk can be incomplete and sampling inefficient. Some formal verification methods suffer from the model precision and other surrogate assumptions. In this paper, we propose a scenario sampling based testing framework that characterizes the overall safety performance of a legged robot by specifying (i) where (in terms of a set of states) the robot is …
Australian Centre for Field Robotics（澳大利亚野外机器人学中心）；University of Sydney（悉尼大学）
The widescale deployment of Autonomous Vehicles (AV) appears to be imminent despite many safety challenges that are yet to be resolved. It is well-known that there are no universally agreed Verification and Validation (VV) methodologies guarantee absolute safety, which is crucial for the acceptance of this technology. The uncertainties in the behaviour of the traffic participants and the dynamic world cause stochastic reactions in advanced autonomous systems. The addition of ML algorithms and probabilistic techniques adds significant complexity to the process for real-world testing when compar…
Australian Centre for Field Robotics (ACFR)（澳大利亚野外机器人中心（ACFR））；University of Sydney（悉尼大学）
The widescale deployment of Autonomous Vehicles (AV) seems to be imminent despite many safety challenges that are yet to be resolved. It is well known that there are no universally agreed Verification and Validation (VV) methodologies to guarantee absolute safety, which is crucial for the acceptance of this technology. Existing standards focus on deterministic processes where the validation requires only a set of test cases that cover the requirements. Modern autonomous vehicles will undoubtedly include machine learning and probabilistic techniques that require a much more comprehensive testin…
UIUC（伊利诺伊大学厄巴纳-香槟分校）；Google Research USA（谷歌美国研究院）；Google Research Accra（谷歌阿克拉研究院）
We consider learning to optimize a classification metric defined by a black-box function of the confusion matrix. Such black-box learning settings are ubiquitous, for example, when the learner only has query access to the metric of interest, or in noisy-label and domain adaptation applications where the learner must evaluate the metric via performance evaluation using a small validation sample. Our approach is to adaptively learn example weights on the training dataset such that the resulting weighted objective best approximates the metric on the validation sample. We show how to model and est…
Autonomous Vehicles (AVs) i.e., self-driving cars, operate in a safety critical domain, since errors in the autonomous driving software can lead to huge losses. Statistically, road intersections which are a part of the AVs operational design domain (ODD), have some of the highest accident rates. Hence, testing AVs to the limits on road intersections and assuring their safety on road intersections is pertinent, and thus the focus of this paper. We present a situation coverage-based (SitCov) AV-testing framework for the verification and validation (V&V) and safety assurance of AVs, developed in …
An Approach to Systematic Data Acquisition and Data-Driven Simulation for the Safety Testing of Automated Driving Functions
Stuttgart Media University（斯图加特媒体大学）；Fraunhofer-Gesellschaft e.V.（弗劳恩霍夫协会）；Fraunhofer EMI（弗劳恩霍夫应用信息技术研究所）；Fraunhofer IVI（弗劳恩霍夫交通与基础设施系统研究所）；Fraunhofer IOSB（弗劳恩霍夫光学、微系统和图像处理研究所）；GOTECH GmbH（高泰克有限公司）；Technische Universität München（慕尼黑工业大学）；Karlsruhe Institute of Technology（卡尔斯鲁厄理工学院）；understandAI GmbH（安代人工智能有限公司）；Porsche Engineering Group GmbH（保时捷工程集团有限公司）
Enhancing Safety Standards in Automated Systems Using Dynamic Bayesian Networks
Aschaffenburg University of Applied Sciences（亚斯法恩堡应用科学大学）；HUGIN EXPERT A/S（HUGIN EXPERT公司）；Department of Computer Science, Aalborg University（计算机科学系，奥胡斯大学）
RWTH Aachen University（亚琛工业大学）；Institute for Automotive Engineering（汽车工程研究所）
Safety assurance of automated driving systems must consider uncertain environment perception. This paper reviews literature addressing how perception testing is realized as part of safety assurance. We focus on testing for verification and validation purposes at the interface between perception and planning, and structure our analysis along the three axes 1) test criteria and metrics, 2) test scenarios, and 3) reference data. Furthermore, the analyzed literature includes related safety standards, safety-independent perception algorithm benchmarking, and sensor modeling. We find that the realiz…
Stanford University（斯坦福大学）；NASA Ames Research Center（美国国家航空航天局埃姆斯研究中心）
Autonomous cyber-physical systems (CPS) can improve safety and efficiency for safety-critical applications, but require rigorous testing before deployment. The complexity of these systems often precludes the use of formal verification and real-world testing can be too dangerous during development. Therefore, simulation-based techniques have been developed that treat the system under test as a black box operating in a simulated environment. Safety validation tasks include finding disturbances in the environment that cause the system to fail (falsification), finding the most-likely failure, and …
Bridging Data-Driven and Knowledge-Driven Approaches for Safety-Critical Scenario Generation in Automated Vehicle Validation
Research Center of Synkrotron, Inc.（同步辐射公司研究中心）；Institute for Interdisciplinary Information Core Technology（跨学科信息核心技术研究所）；Academy of Advanced Interdisciplinary Research, Xidian University（西安电子科技大学先进交叉学科研究院）
A Survey of Safety and Trustworthiness of Large Language Models through the Lens of Verification and Validation
University of Liverpool（利物浦大学）；Purple Mountain Laboratories（紫金山实验室）；Université Grenoble Alpes（格勒诺布尔阿尔卑斯大学）；The University of Manchester（曼彻斯特大学）；KU Leuven（鲁汶大学）
Gaussian Processes (GPs) are widely employed in control and learning because of their principled treatment of uncertainty. However, tracking uncertainty for iterative, multi-step predictions in general leads to an analytically intractable problem. While approximation methods exist, they do not come with guarantees, making it difficult to estimate their reliability and to trust their predictions. In this work, we derive formal probability error bounds for iterative prediction and planning with GPs. Building on GP properties, we bound the probability that random trajectories lie in specific regi…
Iterative Encoding-Decoding VAEs Anomaly Detection in NOAA's DART Time Series: A Machine Learning Approach for Enhancing Data Integrity for NASA's GRACE-FO Verification and Validation
University of California at Los Angeles（加利福尼亚大学洛杉矶分校）；NASA Jet Propulsion Laboratory（美国国家航空航天局喷气推进实验室）
Learning from Risk: LLM-Guided Generation of Safety-Critical Scenarios with Prior Knowledge
Chinese Academy of Sciences, China（中国科学院）；Department of Urban Studies and Planning, Massachusetts Institute of Technology, USA（麻省理工学院城市研究与规划系）；School of Vehicle and Mobility, Tsinghua University, China（清华大学车辆与移动系统学院）
The use of neural networks and reinforcement learning has become increasingly popular in autonomous vehicle control. However, the opaqueness of the resulting control policies presents a significant barrier to deploying neural network-based control in autonomous vehicles. In this paper, we present a reinforcement learning based approach to autonomous vehicle longitudinal control, where the rule-based safety cages provide enhanced safety for the vehicle as well as weak supervision to the reinforcement learning agent. By guiding the agent to meaningful states and actions, this weak supervision im…
