In recent years, curial incidents and accidents have been reported due to un-intended control caused by misjudgment of statistical machine learning (SML), which include deep learning. The international functional safety standards for Electric/Electronic/Programmable (E/E/P) systems have been widely spread to improve safety. However, most of them do not recom-mended to use SML in safety critical systems so far. In practical the new concepts and methods are urgently required to enable SML to be safely used in safety critical systems. In this paper, we organize five kinds of technical safety concepts (TSCs) for SML components toward accordance with functional safety standards. We discuss not only quantitative evaluation criteria, but also development process based on XAI (eXplainable Artificial Intelligence) and Automotive SPICE to improve explainability and reliability in development phase. Fi-nally, we briefly compare the TSCs in cost and difficulty, and expect to en-courage further discussion in many communities and domain.
Bridging Distribution Shift and AI Safety: Conceptual and Methodological Synergies
Center for Data Science, New York University New York New York USA；Computer Science Department, University of California, Santa Barbara Santa Barbara California USA；Department of Electrical；Computer Engineering, University of California, Santa Barbara Santa Barbara California USA；Courant Institute for Mathematical Sciences \& Center for Data Science, New York University New York New York USA；Center for Data Science, New York University；Computer Science Department, University of California, Santa Barbara；Computer Engineering, University of California, Santa Barbara；Courant Institute for Mathematical Sciences \& Center for Data Science, New York University
Generative Grasp Detection and Estimation with Concept Learning-based Safety Criteria
Innovative Factory Solutions（创新工厂解决方案）；German Research Center for Artificial Intelligence (DFKI)（德国人工智能研究中心）；Chair of Machine Tools and Control Systems (WSKL)（机械工具与控制系统教授职位）；Rheinland-Pfälzische Technische Universität (RPTU)（莱茵-瓦尔德技术大学）
Safety Without Semantic Disruptions: Editing-free Safe Image Generation via Context-preserving Dual Latent Reconstruction
University of Western Australia（西澳大学）；University of Melbourne（墨尔本大学）；University of Central Florida（中佛罗里达大学）；Australian National University（澳大利亚国立大学）；Google（谷歌）
The 2025 OpenAI Preparedness Framework does not guarantee any AI risk mitigation practices: a proof-of-concept for affordance analyses of AI safety policies
Agrifood Innovation Institute, Australian National University（澳大利亚国立大学农业食品创新研究院）；MIT FutureTech, Massachusetts Institute of Technology（麻省理工学院未来科技研究所）；The University of Queensland（昆士兰大学）；School of Cybernetics, Australian National University（澳大利亚国立大学控制论学院）；School of Sociology, Australian National University（澳大利亚国立大学社会学学院）；Department of Sociology, Vanderbilt University（范德堡大学社会学系）；Center for Democracy and Technology（民主与技术中心）
本文基于可供性理论分析 OpenAI 的 Preparedness Framework，发现该政策未能有效保证风险缓解，表明 AI 安全需超越行业自律的更稳健治理干预。
Disentangling AI Alignment: A Structured Taxonomy Beyond Safety and Ethics
Responsible AI & Machine Ethics Research Group (RAIME)（负责任的人工智能与机器伦理研究组）；German Research Center for Artificial Intelligence (DFKI)（德国人工智能研究中心）；Saarland Informatics Campus D 3 2（萨尔兰州信息学校园D32）
针对 AI Safety、AI Alignment 与 Machine Ethics 边界模糊的问题，本文提出按目的、范围和构成主体划分的结构化对齐分类法，以促进跨领域整合。
AI Risk Management Should Incorporate Both Safety and Security
Princeton University（普林斯顿大学）；Virginia Tech（弗吉尼亚理工大学）；ETH Zurich（苏黎世联邦理工学院）；ELLIS Institute Tübingen（图宾根ELLIS研究所）；MPI for Intelligent Systems（马克斯·普朗克智能系统研究所）；Stanford University（斯坦福大学）；JPMorgan AI Research（摩根大通AI研究院）；Sony AI（索尼AI研究院）；University of Washington（华盛顿大学）；IBM Research（IBM研究院）；George Washington University（乔治·华盛顿大学）；University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；University of Pennsylvania（宾夕法尼亚大学）；University of Wisconsin, Madison（威斯康星大学麦迪逊分校）；NVIDIA（英伟达公司）；University of Chicago（芝加哥大学）；UC Berkeley（加州大学伯克利分校）
What is Safety? Corporate Discourse, Power, and the Politics of Generative AI Safety
College of Information Sciences and Technology, The Pennsylvania State University（信息科学与技术学院，宾夕法尼亚州立大学）；Max Planck Institute for Security and Privacy（安全与隐私研究所）
This paper considers the design-phase safety analysis of vehicle guidance systems. The proposed approach constructs dynamic fault trees (DFTs) to model a variety of safety concepts and E/E architectures for drive automation. The fault trees can be used to evaluate various quantitative measures by means of model checking. The approach is accompanied by a large-scale evaluation: The resulting DFTs with up to 300 elements constitute larger-than-before DFTs, yet the concepts and architectures can be evaluated in a matter of minutes.
fortiss - Landesforschungsinstitut des Freistaats Bayern（巴伐利亚州fortiss研究机构）
We propose a methodology for designing dependable Artificial Neural Networks (ANN) by extending the concepts of understandability, correctness, and validity that are crucial ingredients in existing certification standards. We apply the concept in a concrete case study in designing a high-way ANN-based motion predictor to guarantee safety properties such as impossibility for the ego vehicle to suggest moving to the right lane if there exists another vehicle on its right.
The specification, design, and assurance of safety encompasses various concepts and best practices, subject of reuse in form of patterns. This work summarizes applied research on such concepts and practices with a focus on the last two decades and on the state-of-the-art of patterns in safety-critical system design and assurance argumentation. We investigate several aspects of such patterns, for example, where and when they are applied, their characteristics and purposes, and how they are related. For each aspect, we provide an overview of relevant studies and synthesize a taxonomy of first pr…
Dealing with Commercial off-the-shelf (COTS) com- ponents is a daily business for avionic system manufacturers. They are necessary ingredients for hardware designs, but are not built in accordance with the avionics consensus standard DO- 254 for Airborne Electronic Hardware (AEH) design. Especially for complex COTS hardware components used in safety critical AEH, like Microcontroller Units (MCUs), additional assurance activities have to be performed. All of them together shall form a convincing confident, that the hardware is safe in its intended operation environment. The focus of DO-254 is o…
The automotive industry is experiencing a transition from assisted to highly automated driving. New concepts for validation of Automated Driving System (ADS) include amongst other a shift from a "technology based" approach to a "scenario based" assessment. The safety validation and type approval process of ADS are seen as the biggest challenges for the automotive industry today. Having in mind a variety of existing white papers, standardization activities and regulatory approaches, manufactures still struggle with selecting the best practices that keep aligned with their Safety Management Syst…
The development of cyber-physical systems can significantly benefit from domain-specific modeling and requires adequate (meta)-modeling frameworks. If such systems are designed for the safety-critical area, the systems must undergo qualification processes defined and monitored by a certification authority. To use the resulting artifacts of modeling tools without further qualification activities, the modeling tool must be additionally qualified. Tool qualification has to be conducted by the tool user and can be assisted by the tool developer by providing qualification artifacts. However, state-…
Safety assurance is a paramount factor in the large-scale deployment of various autonomous systems (e.g., self-driving vehicles). However, the execution of safety engineering practices and processes have been challenged by an increasing complexity of modern safety-critical systems. This attribute has become more critical for autonomous systems that involve artificial intelligence (AI) and data-driven techniques along with the complex interactions of the physical world and digital computing platforms. In this position paper, we highlight some challenges of applying current safety processes to m…
We propose a method for deploying a safety-critical machine-learning component into continuously evolving environments where an increased degree of automation in the engineering process is desired. We associate semantic tags with the safety case argumentation and turn each piece of evidence into a quantitative metric or a logic formula. With proper tool support, the impact can be characterized by a query over the safety argumentation tree to highlight evidence turning invalid. The concept is exemplified using a vision-based emergency braking system of an autonomous guided vehicle for factory a…
Testing of autonomous systems is extremely important as many of them are both safety-critical and security-critical. The architecture and mechanism of such systems are fundamentally different from traditional control software, which appears to operate in more structured environments and are explicitly instructed according to the system design and implementation. To gain a better understanding of autonomous systems practice and facilitate research on testing of such systems, we conducted an exploratory study by synthesizing academic literature with a focus group discussion and interviews with i…
The importance of mission or safety critical software systems in many application domains of embedded systems is continuously growing, and so is the effort and complexity for reliability and safety analysis. Model driven development is currently one of the key approaches to cope with increasing development complexity, in general. Applying similar concepts to reliability, availability, maintainability and safety (RAMS) analysis activities is a promising approach to extend the advantages of model driven development to safety engineering activities aiming at a reduction of development costs, a hi…
