Context: Across different domains, Artificial Neural Networks (ANNs) are used more and more in safety-critical applications in which erroneous outputs of such ANN can have catastrophic consequences. However, the development of such neural networks is still immature and good engineering practices are missing. With that, ANNs are in the same position as software was several decades ago. Today, standards for functional safety, such as ISO 26262 in the automotive domain, require the application of a collection of proven engineering principles and methods in the creation of software to increase its quality and reduce failure rates to an acceptable level. Objective: In the future, such a set of proven engineering methods needs to be established for the development of Artificial Neural Networks to allow their use in safety-critical applications. Method: This work takes a step in this direction by conducting a mapping study to extract challenges faced in the development of ANNs for safety-critical applications and to identify methods that have been used for the hardening of ANNs in such settings. Results: We extracted ten different challenges found to be repeatedly reported in the literature regarding the use of ANNs in critical contexts. All of these challenges are addressed by engineering methods, of which we identified 54 in our study that can be used for the hardening of networks. Conclusions: Various methods have been proposed to overcome the specific challenges of using ANNs in safety-critical applications. On the path towards defining best practices, we envision that future software engineering will need to focus on further investigating these methods and increasing the maturity and understanding of existing approaches, with the goal to develop clear guidance for proper engineering of high-quality ANNs.
Artificial Intelligence, connected products, virtual reality: potential impacts on consumer safety in terms of their physical and psychological ability or well-being
Orphan risks at the frontier of artificial intelligence: What diverging safety and compliance frameworks reveal about how AI companies choose the risks they prioritize
School for the Future of Innovation in Society, Arizona State University(社会创新未来学院,亚利桑那州立大学)
What is Safety? Corporate Discourse, Power, and the Politics of Generative AI Safety
College of Information Sciences and Technology, The Pennsylvania State University(信息科学与技术学院,宾夕法尼亚州立大学);Max Planck Institute for Security and Privacy(安全与隐私研究所)
The Need for Guardrails with Large Language Models in Medical Safety-Critical Settings: An Artificial Intelligence Application in the Pharmacovigilance Ecosystem
Harvard-MIT Department of Health Sciences and Technology(哈佛-麻省理工卫生科学与技术部);GlaxoSmithKline(葛兰素史克公司);London School of Hygiene and Tropical Medicine(伦敦卫生与热带医学院);Harvard T.H. Chan School of Public Health(哈佛陈曾熙公共卫生学院)
Auction-Based Regulation for Artificial Intelligence
Department of Computer Science, University of Maryland, College Park, MD, USA(马里兰大学计算机科学系,College Park, MD, USA);SAP Labs, LLC(SAP实验室);Department of Computer Science, University of Central Florida, FL, USA(佛罗里达大学计算机科学系,FL, USA)