In this paper we discuss how systems with Artificial Intelligence (AI) can undergo safety assessment. This is relevant, if AI is used in safety related applications. Taking a deeper look into AI models, we show, that many models of artificial intelligence, in particular machine learning, are statistical models. Safety assessment would then have t o concentrate on the model that is used in AI, besides the normal assessment procedure. Part of the budget of dangerous random failures for the relevant safety integrity level needs to be used for the probabilistic faulty behavior of the AI system. We demonstrate our thoughts with a simple example and propose a research challenge that may be decisive for the use of AI in safety related systems.
Idioms are small, reusable Bayesian network (BN) fragments that represent generic types of uncertain reasoning. This paper shows how idioms can be used to build causal BNs for product safety and risk assessment that use a combination of data and knowledge. We show that the specific product safety idioms that we introduce are sufficient to build full BN models to evaluate safety and risk for a wide range of products. The resulting models can be used by safety regulators and product manufacturers even when there are limited (or no) product testing data.
Here, I review current state-of-the-arts in many areas of AI to estimate when it's reasonable to expect human level AI development. Predictions of prominent AI researchers vary broadly from very pessimistic predictions of Andrew Ng to much more moderate predictions of Geoffrey Hinton and optimistic predictions of Shane Legg, DeepMind cofounder. Given huge rate of progress in recent years and this broad range of predictions of AI experts, AI safety questions are also discussed.
University of New South Wales（新南威尔士大学）；BeU Research Group（BeU研究组）；University of Glasgow（格拉斯哥大学）
With the rapid adoption of Internet technologies, digital footprints have become ubiquitous and versatile to revolutionise the financial industry in digital transformation. This paper takes initiatives to investigate a new paradigm of the unified credit assessment with the use of federated artificial intelligence. We conceptualised digital human representation which consists of social, contextual, financial and technological dimensions to assess the commercial creditworthiness and social reputation of both banked and unbanked individuals. A federated artificial intelligence platform is propose…
LAAS-CNRS（法国国家科学研究中心系统分析与架构实验室）；Université de Toulouse（图卢兹大学）
Building a safety case is a common approach to make expert judgement explicit about safety of a system. The issue of confidence in such argumentation is still an open research field. Providing quantitative estimation of confidence is an interesting approach to manage complexity of arguments. This paper explores the main current approaches, and proposes a new model for quantitative confidence estimation based on Belief Theory for its definition, and on Bayesian Belief Networks for its propagation in safety case networks.
Cornell University（康奈尔大学）；University of Montreal（蒙特利尔大学）；University of Louisville（路易斯维尔大学）
With almost daily improvements in capabilities of artificial intelligence it is more important than ever to develop safety software for use by the AI research community. Building on our previous work on AI Containment Problem we propose a number of guidelines which should help AI safety researchers to develop reliable sandboxing software for intelligent programs of all levels. Such safety container software will make it possible to study and analyze intelligent artificial agent while maintaining certain level of safety against information leakage, social engineering attacks and cyberattacks fr…
Microsoft Corporation（微软公司）；University of Louisville（路易斯维尔大学）
In this work, we present and analyze reported failures of artificially intelligent systems and extrapolate our analysis to future AIs. We suggest that both the frequency and the seriousness of future AI failures will steadily increase. AI Safety can be improved based on ideas developed by cybersecurity experts. For narrow AIs safety failures are at the same, moderate, level of criticality as in cybersecurity, however for general AI, failures have a fundamentally different impact. A single failure of a superintelligent system may cause a catastrophic event without a chance for recovery. The goa…
Document Retrieval Augmented Fine-Tuning (DRAFT) for safety-critical software assessments
Digital Transit Limited, 3M Buckley Innovation Centre, UK, HD1 3BD（数字交通有限公司，3M Buckley创新中心，英国，HD1 3BD）；Department of Computer Science, University of Huddersfield, UK, HD1 3DH（计算机科学系，赫德瑟菲尔德大学，英国，HD1 3DH）
An increasing number of artists use Ai in their creative practices (Creative-Ai) and their works have by now become visible at prominent art venues. The research community has, on the other hand, recognized that there are sustainability concerns of using Ai technologies related to, for instance, energy consumption and the increasing size and complexity of models. These two conflicting trajectories constitute the starting point of our research. Here, we discuss insights from our currently on-going fieldwork research and outline considerations for drawing various limitations in sustainability as…
Several different approaches exist for ensuring the safety of future Transformative Artificial Intelligence (TAI) or Artificial Superintelligence (ASI) systems, and proponents of different approaches have made different and debated claims about the importance or usefulness of their work in the near term, and for future systems. Highly Reliable Agent Designs (HRAD) is one of the most controversial and ambitious approaches, championed by the Machine Intelligence Research Institute, among others, and various arguments have been made about whether and how it reduces risks from future AI systems. I…
Evaluating the potential of a prospective candidate is a common task in multiple decision-making processes in different industries. We refer to a prospect as something or someone that could potentially produce positive results in a given context, e.g., an area where an oil company could find oil, a compound that, when synthesized, results in a material with required properties, and so on. In many contexts, assessing the Probability of Success (PoS) of prospects heavily depends on experts' knowledge, often leading to biased and inconsistent assessments. We have developed the framework named KAR…
Inserm（法国国家健康与医学研究院）；Université Paul Sabatier Toulouse 3（图卢兹第三大学）；GIS Genotoul（Genotoul科学研究集团）；Université Fédérale de Toulouse（图卢兹联邦大学）；Groupement d’Intérêt Public e.santé Occitanie（奥克西塔尼电子健康公共利益集团）；IRIT-CNRS（图卢兹信息研究所-法国国家科学研究中心）
This paper proposes a comprehensive analysis of existing concepts coming from different disciplines tackling the notion of intelligence, namely psychology and engineering, and from disciplines aiming to regulate AI innovations, namely AI ethics and law. The aim is to identify shared notions or discrepancies to consider for qualifying AI systems. Relevant concepts are integrated into a matrix intended to help defining more precisely when and how computing tools (programs or devices) may be qualified as AI while highlighting critical features to serve a specific technical, ethical and legal asse…
University of São Paulo（圣保罗大学）；Ericsson AB（爱立信公司）；Ericsson Telecommunication S.A.（爱立信电信公司）
Autonomous Vehicles (AV) are expected to bring considerable benefits to society, such as traffic optimization and accidents reduction. They rely heavily on advances in many Artificial Intelligence (AI) approaches and techniques. However, while some researchers in this field believe AI is the core element to enhance safety, others believe AI imposes new challenges to assure the safety of these new AI-based systems and applications. In this non-convergent context, this paper presents a systematic literature review to paint a clear picture of the state of the art of the literature in AI on AV saf…
Clemson University（克莱姆森大学）；Northern Arizona University（北亚利桑那大学）；The University of Arizona（亚利桑那大学）；Arizona Department of Transportation（亚利桑那州交通局）；Exponent Co（Exponent公司）
Driving safety analysis has recently experienced unprecedented improvements thanks to technological advances in precise positioning sensors, artificial intelligence (AI)-based safety features, autonomous driving systems, connected vehicles, high-throughput computing, and edge computing servers. Particularly, deep learning (DL) methods empowered volume video processing to extract safety-related features from massive videos captured by roadside units (RSU). Safety metrics are commonly used measures to investigate crashes and near-conflict events. However, these metrics provide limited insight in…
Artificial intelligence is known as the most effective technological field for rapid developments shaping the future of the world. Even today, it is possible to see intense use of intelligence systems in all fields of the life. Although advantages of the Artificial Intelligence are widely observed, there is also a dark side employing efforts to design hacking oriented techniques against Artificial Intelligence. Thanks to such techniques, it is possible to trick intelligent systems causing directed results for unsuccessful outputs. That is critical for also cyber wars of the future as it is pre…
Montgomery County, Maryland Community Emergency Response Team（马里兰州蒙哥马利县社区应急响应队）
Rapid damage assessment is one of the core tasks that response organizations perform at the onset of a disaster to understand the scale of damage to infrastructures such as roads, bridges, and buildings. This work analyzes the usefulness of social media imagery content to perform rapid damage assessment during a real-world disaster. An automatic image processing system, which was activated in collaboration with a volunteer response organization, processed ~280K images to understand the extent of damage caused by the disaster. The system achieved an accuracy of 76% computed based on the feedbac…
Dublin College University（都柏林学院大学）；Ulster University（阿尔斯特大学）；University of Science and Technology Beijing（北京科技大学）
Death by suicide is the seventh leading death cause worldwide. The recent advancement in Artificial Intelligence (AI), specifically AI applications in image and voice processing, has created a promising opportunity to revolutionize suicide risk assessment. Subsequently, we have witnessed fast-growing literature of research that applies AI to extract audiovisual non-verbal cues for mental illness assessment. However, the majority of the recent works focus on depression, despite the evident difference between depression symptoms and suicidal behavior and non-verbal cues. This paper reviews recen…
This chapter formulates seven lessons for preventing harm in artificial intelligence (AI) systems based on insights from the field of system safety for software-based automation in safety-critical domains. New applications of AI across societal domains and public organizations and infrastructures come with new hazards, which lead to new forms of harm, both grave and pernicious. The text addresses the lack of consensus for diagnosing and eliminating new AI system hazards. For decades, the field of system safety has dealt with accidents and harm in safety-critical systems governed by varying deg…
TSHA: A Benchmark for Visual Language Models in Trustworthy Safety Hazard Assessment Scenarios
City University of Hong Kong（香港城市大学）；East China Normal University（华东师范大学）；University of Western Australia（西澳大学）；Shanghai AI Laboratory（上海人工智能实验室）；Shanghai Development Center of Computer Software Technology（上海计算机软件技术开发中心）；Zhejiang Gongshang University（浙江工商大学）
We present CAISAR, an open-source platform under active development for the characterization of AI systems' robustness and safety. CAISAR provides a unified entry point for defining verification problems by using WhyML, the mature and expressive language of the Why3 verification platform. Moreover, CAISAR orchestrates and composes state-of-the-art machine learning verification tools which, individually, are not able to efficiently handle all problems but, collectively, can cover a growing number of properties. Our aim is to assist, on the one hand, the V\&V process by reducing the burden of ch…
Can Artificial Intelligence Support Healthcare and Mental Health Through Early Cyberbullying Detection ? The Impact of Emotion-Aware AI on Proactive Online Safety
Faculty of Computer Science, University of New Brunswick（新不伦瑞克大学计算机科学学院）；Khoury College of Computer Sciences, Northeastern University（东北大学库里计算机科学学院）
Ericsson AB（爱立信公司）；Ericsson Telecomunicações S/A（爱立信电信公司）；Mälardalen University（梅拉达伦大学）
The recent development of artificial intelligence (AI) has increased the interest of researchers and practitioners towards applying its techniques into multiple domains like automotive, health care and air space to achieve automation. Combined to these applications, the attempt to use AI techniques into carrying out safety issues is momentarily at a progressive state. As AI problems are getting even more complex, large processing power is demanded for safety-critical systems to fulfill real-time requirements. These challenges can be solved through edge or cloud computing, which makes the commu…
University of Liverpool（利物浦大学）；Heriot-Watt University（赫瑞-瓦特大学）；Defence Science and Technology Laboratory（国防科学技术实验室）；Imperial College London（帝国理工学院）
The increasing use of Machine Learning (ML) components embedded in autonomous systems -- so-called Learning-Enabled Systems (LESs) -- has resulted in the pressing need to assure their functional safety. As for traditional functional safety, the emerging consensus within both, industry and academia, is to use assurance cases for this purpose. Typically assurance cases support claims of reliability in support of safety, and can be viewed as a structured way of organising arguments and evidence generated from safety analysis and reliability modelling activities. While such assurance activities ar…
A Different Approach to AI Safety: Proceedings from the Columbia Convening on Openness in Artificial Intelligence and AI Safety
Columbia University（哥伦比亚大学）；Mozilla（谋智公司）；Ai2（艾伦人工智能研究所）；Google DeepMind（谷歌DeepMind）；HuggingFace；Princeton CITP（普林斯顿大学信息技术政策中心）；AI Now Institute（AI Now研究所）；Stanford CRFM（斯坦福大学基础模型研究中心）；Mistral；Collective Intelligence Project（集体智能项目）；EleutherAI；Meta；Partnership on AI（人工智能合作组织）；Taiwan Digital Affairs（台湾数位发展部）；ROOST
Artificial Intelligence for CRISPR Guide RNA Design: Explainable Models and Off-Target Safety
Department of Computer Engineering, Ma.C., Islamic Azad University（计算机工程系，伊斯兰阿兹德大学）；Department of Biological Sciences and Technologies, Faculty of Basic Sciences, Islamic Azad University（基础科学学院生物科学与技术系，伊斯兰阿兹德大学）
Paris School of Business（巴黎商学院）；Usak University（乌沙克大学）
Artificial Intelligence (AI) is an effective science which employs strong enough approaches, methods, and techniques to solve unsolvable real world based problems. Because of its unstoppable rise towards the future, there are also some discussions about its ethics and safety. Shaping an AI friendly environment for people and a people friendly environment for AI can be a possible answer for finding a shared context of values for both humans and robots. In this context, objective of this paper is to address the ethical issues of AI and explore the moral dilemmas that arise from ethical algorithm…
This literature review identifies indicators that associate with higher impact or higher quality research from article text (e.g., titles, abstracts, lengths, cited references and readability) or metadata (e.g., the number of authors, international or domestic collaborations, journal impact factors and authors' h-index). This includes studies that used machine learning techniques to predict citation counts or quality scores for journal articles or conference papers. The literature review also includes evidence about the strength of association between bibliometric indicators and quality score …
Conventional methods for visual assessment of civil infrastructures have certain limitations, such as subjectivity of the collected data, long inspection time, and high cost of labor. Although some new technologies i.e. robotic techniques that are currently in practice can collect objective, quantified data, the inspectors own expertise is still critical in many instances since these technologies are not designed to work interactively with human inspector. This study aims to create a smart, human centered method that offers significant contributions to infrastructure inspection, maintenance, m…
The AI Robotics Ethics Society (AIRES) is a non-profit organization founded in 2018 by Aaron Hui to promote awareness and the importance of ethical implementation and regulation of AI. AIRES is now an organization with chapters at universities such as UCLA (Los Angeles), USC (University of Southern California), Caltech (California Institute of Technology), Stanford University, Cornell University, Brown University, and the Pontifical Catholic University of Rio Grande do Sul (Brazil). AIRES at PUCRS is the first international chapter of AIRES, and as such, we are committed to promoting and enhan…
