With the growing capabilities of intelligent systems, the integration of artificial intelligence (AI) and robots in everyday life is increasing. However, when interacting in such complex human environments, the failure of intelligent systems, such as robots, can be inevitable, requiring recovery assistance from users. In this work, we develop automated, natural language explanations for failures encountered during an AI agents' plan execution. These explanations are developed with a focus of helping non-expert users understand different point of failures to better provide recovery assistance. Specifically, we introduce a context-based information type for explanations that can both help non-expert users understand the underlying cause of a system failure, and select proper failure recoveries. Additionally, we extend an existing sequence-to-sequence methodology to automatically generate our context-based explanations. By doing so, we are able develop a model that can generalize context-based explanations over both different failure types and failure scenarios.
While recent advances in AI-based automated decision-making have shown many benefits for businesses and society, they also come at a cost. It has for long been known that a high level of automation of decisions can lead to various drawbacks, such as automation bias and deskilling. In particular, the deskilling of knowledge workers is a major issue, as they are the same people who should also train, challenge and evolve AI. To address this issue, we conceptualize a new class of DSS, namely Intelligent Decision Assistance (IDA) based on a literature review of two different research streams -- DS…
With the growing capabilities of intelligent systems, the integration of robots in our everyday life is increasing. However, when interacting in such complex human environments, the occasional failure of robotic systems is inevitable. The field of explainable AI has sought to make complex-decision making systems more interpretable but most existing techniques target domain experts. On the contrary, in many failure cases, robots will require recovery assistance from non-expert users. In this work, we introduce a new type of explanation, that explains the cause of an unexpected failure during an…
Kalasalingam Academy for Research and Education（卡萨斯林甘研究与教育学院）；GE Corporate Research Center（通用电气企业研究中心）；SINTEF；Norwegian University of Science and Technology（挪威科技大学）；SUNY Polytechnic Institute（纽约州立大学理工学院）
This paper presents an eXplainable Fault Detection and Diagnosis System (XFDDS) for incipient faults in PV panels. The XFDDS is a hybrid approach that combines the model-based and data-driven framework. Model-based FDD for PV panels lacks high fidelity models at low irradiance conditions for detecting incipient faults. To overcome this, a novel irradiance based three diode model (IB3DM) is proposed. It is a nine parameter model that provides higher accuracy even at low irradiance conditions, an important aspect for detecting incipient faults from noise. To exploit PV data, extreme gradient boo…
With the advent of Industry 4.0, Data Science and Explainable Artificial Intelligence (XAI) has received considerable intrest in recent literature. However, the entry threshold into XAI, in terms of computer coding and the requisite mathematical apparatus, is really high. For fault diagnosis of steel plates, this work reports on a methodology of incorporating XAI based insights into the Data Science process of development of high precision classifier. Using Synthetic Minority Oversampling Technique (SMOTE) and notion of medoids, insights from XAI tools viz. Ceteris Peribus profiles, Partial De…
Federal University of Uberlândia（乌贝兰迪亚联邦大学）；University of Padova（帕多瓦大学）；Federal University of São João del Rei（圣若昂德尔雷伊联邦大学）
Artificial Intelligence (AI) is one of the approaches that has been proposed to analyze the collected data (e.g., vibration signals) providing a diagnosis of the asset's operating condition. It is known that models trained with labeled data (supervised) achieve excellent results, but two main problems make their application in production processes difficult: (i) impossibility or long time to obtain a sample of all operational conditions (since faults seldom happen) and (ii) high cost of experts to label all acquired data. Another limitating factor for the applicability of AI approaches in this…
Assessing the Real-World Utility of Explainable AI for Arousal Diagnostics: An Application-Grounded User Study
IT-Designers Gruppe Esslingen am Neckar（埃斯林根内卡尔IT设计组）；University of Tübingen（图宾根大学）；University of Giessen（吉森大学）；Klinikum Esslingen, Klinik für Kardiologie, Pneumologie und Angilologie（埃斯林根克林克医院，心内科、呼吸科和内科）；Technical University of Munich（慕尼黑技术大学）
When interacting in unstructured human environments, occasional robot failures are inevitable. When such failures occur, everyday people, rather than trained technicians, will be the first to respond. Existing natural language explanations hand-annotate contextual information from an environment to help everyday people understand robot failures. However, this methodology lacks generalizability and scalability. In our work, we introduce a more generalizable semantic explanation framework. Our framework autonomously captures the semantic information in a scene to produce semantically descriptive…
Explainable AI as a Double-Edged Sword in Dermatology: The Impact on Clinicians versus The Public
Columbia University（哥伦比亚大学）；Cornell University（康奈尔大学）；MIT（麻省理工学院）；Johns Hopkins University（约翰斯·霍普金斯大学）；Northeast Dermatology Associates（东北皮肤科协会）；Stanford University（斯坦福大学）；University of Utah（犹他大学）；OSF Healthcare（OSF医疗保健）；Columbia University Vagelos College of Physicians and Surgeons（哥伦比亚大学瓦格洛斯内科与外科医学院）；Centaur Labs；MedShr；King’s College Hospital, London（伦敦国王学院医院）；Pathway；X=Primary Care；Northwestern（西北大学）；Medical University of Vienna（维也纳医科大学）
Explainable machine learning (ML) enables human learning from ML, human appeal of automated model decisions, regulatory compliance, and security audits of ML models. Explainable ML (i.e. explainable artificial intelligence or XAI) has been implemented in numerous open source and commercial packages and explainable ML is also an important, mandatory, or embedded aspect of commercial predictive modeling in industries like financial services. However, like many technologies, explainable ML can be misused, particularly as a faulty safeguard for harmful black-boxes, e.g. fairwashing or scaffolding,…
Explainable AI for Multivariate Time Series Pattern Exploration: Latent Space Visual Analytics with Temporal Fusion Transformer and Variational Autoencoders in Power Grid Event Diagnosis
Queen’s University Belfast（贝尔法斯特女王大学）；King’s College London（伦敦国王学院）；University of Liverpool（利物浦大学）；University of Oxford（牛津大学）
The black-box nature of deep neural networks (DNNs) makes it impossible to understand why a particular output is produced, creating demand for "Explainable AI". In this paper, we show that statistical fault localization (SFL) techniques from software engineering deliver high quality explanations of the outputs of DNNs, where we define an explanation as a minimal subset of features sufficient for making the same decision as for the original input. We present an algorithm and a tool called DeepCover, which synthesizes a ranking of the features of the inputs using SFL and constructs explanations …
The Johns Hopkins Applied Physics Laboratory（约翰斯·霍普金斯应用物理实验室）
We propose a novel approach to explainable AI (XAI) based on the concept of "instruction" from neural networks. In this case study, we demonstrate how a superhuman neural network might instruct human trainees as an alternative to traditional approaches to XAI. Specifically, an AI examines human actions and calculates variations on the human strategy that lead to better performance. Experiments with a JHU/APL-developed AI player for the cooperative card game Hanabi suggest this technique makes unique contributions to explainability while improving human performance. One area of focus for Instru…
Cleveland Clinic London Hospital（伦敦克利夫兰诊所医院）；Mid South Essex NHS Foundation Trust（中埃塞克斯南部国民保健服务基金会信托）；King’s College Hospital NHS Foundation Trust（国王学院医院国民保健服务基金会信托）；London South Bank University（伦敦南岸大学）
Objectives: The present study evaluated the impact of a commercially available explainable AI algorithm in augmenting the ability of clinicians to identify lung cancer on chest X-rays (CXR). Design: This retrospective study evaluated the performance of 11 clinicians for detecting lung cancer from chest radiographs, with and without assistance from a commercially available AI algorithm (red dot, Behold.ai) that predicts suspected lung cancer from CXRs. Clinician performance was evaluated against clinically confirmed diagnoses. Setting: The study analysed anonymised patient data from an NHS hosp…
S2S-FDD: Bridging Industrial Time Series and Natural Language for Explainable Zero-shot Fault Diagnosis
1. State Key Laboratory of Industrial Control Technology, College of Control Science；Engineering Zhejiang University Hangzhou, China；Engineering Zhejiang University 2. School of Information；Electrical Engineering, Zhejiang University City College Hangzhou, China
CardioForest: An Explainable Ensemble Learning Model for Automatic Wide QRS Complex Tachycardia Diagnosis from ECG
School of Artificial Intelligence and Computer Science（人工智能与计算机科学学院）；Nantong University（南通大学）；College of Information Science and Engineering（信息科学与工程学院）；Henan University of Technology（河南理工大学）；Affiliated Hospital of Nantong University（南通大学附属医院）
The transformation of conventional power networks into smart grids with the heavy penetration level of renewable energy resources, particularly grid-connected Photovoltaic (PV) systems, has increased the need for efficient fault identification systems. Malfunctioning any single component in grid-connected PV systems may lead to grid instability and other serious consequences, showing that a reliable fault identification system is the utmost requirement for ensuring operational integrity. Therefore, this paper presents a novel fault identification approach based on statistical signatures of PV …
UCLA（加州大学洛杉矶分校）；University of Michigan（密歇根大学）；Oregon State University（俄勒冈州立大学）；Beijing Institute for General AI (BIGAI)（北京通用人工智能研究院）；Tsinghua University（清华大学）；Peking University（北京大学）
We propose CX-ToM, short for counterfactual explanations with theory-of mind, a new explainable AI (XAI) framework for explaining decisions made by a deep convolutional neural network (CNN). In contrast to the current methods in XAI that generate explanations as a single shot response, we pose explanation as an iterative communication process, i.e. dialog, between the machine and human user. More concretely, our CX-ToM framework generates sequence of explanations in a dialog by mediating the differences between the minds of machine and human user. To do this, we use Theory of Mind (ToM) which …
School of Mechanical Engineering, Federal University of Uberlândia（Uberlândia联邦大学机械工程学院）；University of Padova（帕多瓦大学）；Federal University of São João del Rei（圣若昂德尔雷伊联邦大学）
The monitoring of rotating machinery is an essential task in today's production processes. Currently, several machine learning and deep learning-based modules have achieved excellent results in fault detection and diagnosis. Nevertheless, to further increase user adoption and diffusion of such technologies, users and human experts must be provided with explanations and insights by the modules. Another issue is related, in most cases, with the unavailability of labeled historical data that makes the use of supervised models unfeasible. Therefore, a new approach for fault detection and diagnosis…
University of South Carolina（南卡罗来纳大学）；Artificial Intelligence Institute（人工智能研究所）
Improving the performance and natural language explanations of deep learning algorithms is a priority for adoption by humans in the real world. In several domains, such as healthcare, such technology has significant potential to reduce the burden on humans by providing quality assistance at scale. However, current methods rely on the traditional pipeline of predicting labels from data, thus completely ignoring the process and guidelines used to obtain the labels. Furthermore, post hoc explanations on the data to label prediction using explainable AI (XAI) models, while satisfactory to computer…
Confident Teacher, Confident Student? A Novel User Study Design for Investigating the Didactic Potential of Explanations and their Impact on Uncertainty
Berliner Hochschule für Technik（柏林应用科学大学）；Einstein Center Digital Future（爱因斯坦数字未来中心）
University of Trento（特伦托大学）；University of Twente（特温特大学）
Many industrial sectors have been collecting big sensor data. With recent technologies for processing big data, companies can exploit this for automatic failure detection and prevention. We propose the first completely automated method for failure analysis, machine-learning fault trees from raw observational data with continuous variables. Our method scales well and is tested on a real-world, five-year dataset of domestic heater operations in The Netherlands, with 31 million unique heater-day readings, each containing 27 sensor and 11 failure variables. Our method builds on two previous proced…
Technion - I.I.T（以色列理工学院）；University of Haifa（海法大学）
As reinforcement learning methods increasingly amass accomplishments, the need for comprehending their solutions becomes more crucial. Most explainable reinforcement learning (XRL) methods generate a static explanation depicting their developers' intuition of what should be explained and how. In contrast, literature from the social sciences proposes that meaningful explanations are structured as a dialog between the explainer and the explainee, suggesting a more active role for the user and her communication with the agent. In this paper, we present ASQ-IT -- an interactive tool that presents …
Preemptively Pruning Clever-Hans Strategies in Deep Neural Networks
Technische Universität Berlin（柏林工业大学）；Berlin Institute for the Foundations of Learning and Data – BIFOLD（柏林学习与数据基础研究所（BIFOLD））；Korea University（高丽大学）；Max Planck Institute for Informatics（马克斯·普朗克信息学研究所）；Google DeepMind（谷歌DeepMind）；Freie Universität Berlin（柏林自由大学）
