Deep learning models have recently demonstrated remarkable results in a variety of tasks, which is why they are being increasingly applied in high-stake domains, such as industry, medicine, and finance. Considering that automatic predictions in these domains might have a substantial impact on the well-being of a person, as well as considerable financial and legal consequences to an individual or a company, all actions and decisions that result from applying these models have to be accountable. Given that a substantial amount of data that is collected in high-stake domains are in the form of time series, in this paper we examine the current state of eXplainable AI (XAI) methods with a focus on approaches for opening up deep learning black boxes for the task of time series classification. Finally, our contribution also aims at deriving promising directions for future work, to advance XAI for deep learning on time series data.
In this work, we report the practical and theoretical aspects of Explainable AI (XAI) identified in some fundamental literature. Although there is a vast body of work on representing the XAI backgrounds, most of the corpuses pinpoint a discrete direction of thoughts. Providing insights into literature in practice and theory concurrently is still a gap in this field. This is important as such connection facilitates a learning process for the early stage XAI researchers and give a bright stand for the experienced XAI scholars. Respectively, we first focus on the categories of black-box explanati…
A state-of-the-art systematic review on XAI applied to Prognostic and Health Management (PHM) of industrial asset is presented. This work provides an overview of the general trend of XAI in PHM, answers the question of accuracy versus explainability, the extent of human involvement, the explanation assessment and uncertainty quantification in PHM-XAI domain. Research articles associated with the subject, from 2015 to 2021 were selected from five known databases following PRISMA guidelines. Data was then extracted from the selected articles and examined. Several findings were synthesized. First…
The advancements in deep learning-based methods for visual perception tasks have seen astounding growth in the last decade, with widespread adoption in a plethora of application areas from autonomous driving to clinical decision support systems. Despite their impressive performance, these deep learning-based models remain fairly opaque in their decision-making process, making their deployment in human-critical tasks a risky endeavor. This in turn makes understanding the decisions made by these models crucial for their reliable deployment. Explainable AI (XAI) methods attempt to address this by…
Center for Artificial Intelligence Research（人工智能研究中心）；University of Agder（阿格德大学）
The past decade has seen significant progress in artificial intelligence (AI), which has resulted in algorithms being adopted for resolving a variety of problems. However, this success has been met by increasing model complexity and employing black-box AI models that lack transparency. In response to this need, Explainable AI (XAI) has been proposed to make AI more transparent and thus advance the adoption of AI in critical domains. Although there are several reviews of XAI topics in the literature that identified challenges and potential research directions in XAI, these challenges and resear…
Explainable Artificial Intelligence and Multicollinearity : A Mini Review of Current Approaches
University of Leicester（莱斯特大学）；William Harvey Research Institute（威廉·哈维研究所）；Queen Mary University of London（伦敦玛丽女王大学）；Barts Health NHS Trust（巴特健康 NHS 信托）；St Bartholomew’s Hospital（圣巴塞洛缪医院）；University of Zakho（扎胡大学）
Are we still able to recognize pearls? Machine-driven peer review and the risk to creativity: An explainable RAG-XAI detection framework with markers extraction
Bucharest University of Economic Studies（布加勒斯特经济研究大学）
School of Computer Science, College of Science and Health, Technological University Dublin（都柏林理工大学科学与健康学院计算机科学学院）；Technological University Dublin（都柏林理工大学）
Explainable Artificial Intelligence (XAI) has experienced a significant growth over the last few years. This is due to the widespread application of machine learning, particularly deep learning, that has led to the development of highly accurate models but lack explainability and interpretability. A plethora of methods to tackle this problem have been proposed, developed and tested. This systematic review contributes to the body of knowledge by clustering these methods with a hierarchical classification system with four main clusters: review articles, theories and notions, methods and their ev…
A Brief Review of Explainable Artificial Intelligence in Healthcare
Dalhousie University（达尔豪斯大学）；Deakin University（迪肯大学）；TU Wien University（维也纳技术大学）；University of Kotli Azad Jammu and Kashmir（科特利阿扎德查谟和克什米尔大学）；Dibrugarh University（迪布鲁格尔大学）；Al-Balqa Applied University（巴尔卡应用大学）；The University of Jordan（约旦大学）；University of Granada（格拉纳达大学）；Jan Evangelista Purkyně University in Ústí nad Labem（拉贝河畔乌斯季扬·埃万杰利斯塔·普尔基涅大学）；Charles University（查理大学）；Harvard University（哈佛大学）；University of Florida（佛罗里达大学）
On the Design and Evaluation of Human-centered Explainable AI Systems: A Systematic Review and Taxonomy
Dresden University of Technology（德累斯顿技术大学）；Charité Berlin（柏林夏里特医学院）；Department Speculative Transformation（推测转化部门）；Chair of Engineering Psychology and Applied Cognitive Research（工程心理学与应用认知研究教授职位）
Nanyang Technological University（南洋理工大学）；Alibaba Group Holding Limited（阿里巴巴集团控股有限公司）
Recently, artificial intelligence and machine learning in general have demonstrated remarkable performances in many tasks, from image processing to natural language processing, especially with the advent of deep learning. Along with research progress, they have encroached upon many different fields and disciplines. Some of them require high level of accountability and thus transparency, for example the medical sector. Explanations for machine decisions and predictions are thus needed to justify their reliability. This requires greater interpretability, which often means we need to understand t…
When people buy products online, they primarily base their decisions on the recommendations of others given in online reviews. The current work analyzed these online reviews by sentiment analysis and used the extracted sentiments as features to predict the product ratings by several machine learning algorithms. These predictions were disentangled by various meth-ods of explainable AI (XAI) to understand whether the model showed any bias during prediction. Study 1 benchmarked these algorithms (knn, support vector machines, random forests, gradient boosting machines, XGBoost) and identified rand…
Explainable Artificial Intelligence: A Survey of Needs, Techniques, Applications, and Future Direction
University of Gondar（贡德尔大学）；College of Engineering and Applied Science, University of Colorado, Colorado Springs（科罗拉多大学科罗拉多斯普林斯分校工程与应用科学学院）；Can Tho University（芹苴大学）；Ewha Womans University（梨花女子大学）；University of Colorado, Colorado Springs（科罗拉多大学科罗拉多斯普林斯分校）
Meta Reality Labs（Meta现实实验室）；Purdue（普渡大学）；Aarhus Univ（奥胡斯大学）；Dartmouth（达特茅斯学院）；Cambridge（剑桥大学）
Explainable AI (XAI) has established itself as an important component of AI-driven interactive systems. With Augmented Reality (AR) becoming more integrated in daily lives, the role of XAI also becomes essential in AR because end-users will frequently interact with intelligent services. However, it is unclear how to design effective XAI experiences for AR. We propose XAIR, a design framework that addresses "when", "what", and "how" to provide explanations of AI output in AR. The framework was based on a multi-disciplinary literature review of XAI and HCI research, a large-scale survey probing …
Cybersecurity vendors consistently apply AI (Artificial Intelligence) to their solutions and many cybersecurity domains can benefit from AI technology. However, black-box AI techniques present some difficulties in comprehension and adoption by its operators, given that their decisions are not always humanly understandable (as is usually the case with deep neural networks, for example). Since it aims to make the operation of AI algorithms more interpretable for its users and developers, XAI (eXplainable Artificial Intelligence) can be used to address this issue. Through a systematic literature …
LUT University（拉彭兰塔-拉赫蒂工业大学）；Norwegian University of Science and Technology（挪威科技大学）；SINTEF Digital（挪威科技工业研究院数字研究所）
The final search query for the Systematic Literature Review (SLR) was conducted on 15th July 2022. Initially, we extracted 1707 journal and conference articles from the Scopus and Web of Science databases. Inclusion and exclusion criteria were then applied, and 58 articles were selected for the SLR. The findings show four dimensions that shape the AI explanation, which are format (explanation representation format), completeness (explanation should contain all required information, including the supplementary information), accuracy (information regarding the accuracy of the explanation), and c…
Rice University（莱斯大学）；Meta Platforms, Inc.（元平台公司）；New Jersey Institute of Technology（新泽西理工学院）
Recently, there has been a growing demand for the deployment of Explainable Artificial Intelligence (XAI) algorithms in real-world applications. However, traditional XAI methods typically suffer from a high computational complexity problem, which discourages the deployment of real-time systems to meet the time-demanding requirements of real-world scenarios. Although many approaches have been proposed to improve the efficiency of XAI methods, a comprehensive understanding of the achievements and challenges is still needed. To this end, in this paper we provide a review of efficient XAI. Specifi…
Applications of Explainable artificial intelligence in Earth system science
Southern Marine Science and Engineering Guangdong Laboratory (Zhuhai)（南方海洋科学与工程广东省实验室（珠海））；Guangdong Province Key Laboratory for Climate Change and Natural Disaster Studies（广东省气候变化与自然灾害研究重点实验室）；Max Planck Institute for Biogeochemistry（马克斯·普朗克生物地球化学研究所）；Shanwei Meteorological Service（汕尾市气象局）；College of Computer Science and Technology, Changchun Normal University（长春师范大学计算机科学与技术学院）；ELLIS Unit Jena（ELLIS耶拿单元）
Breast Cancer Diagnosis: A Comprehensive Exploration of Explainable Artificial Intelligence (XAI) Techniques
Canadian Institute of Cybersecurity (CIC)（加拿大网络安全研究所）；University of New Brunswick（新不伦瑞克大学）；Dipartimento di Informatica, Università di Verona（威尼斯大学信息学院）；Università di Verona（威尼斯大学）；Department of Computer Science, School of Mathematics and Computer Science, Institute of Business Administration（计算机科学系，数学与计算机科学学院，商学院）；LIRIS, CNRS, Université Claude Bernard Lyon 1（LIRIS，国家科学研究中心，克莱尔伯恩大学 Lyon 1 分校）；LHC, UMR5516, Université de Saint-Etienne（LHC，UMR5516，圣艾蒂安大学）
J. P. Morgan AI Research（摩根大通人工智能研究院）；J. P. Morgan（摩根大通）
Regulators have signalled an interest in adopting explainable AI(XAI) techniques to handle the diverse needs for model governance, operational servicing, and compliance in the financial services industry. In this short overview, we review the recent technical literature in XAI and argue that based on our current understanding of the field, the use of XAI techniques in practice necessitate a highly contextualized approach considering the specific needs of stakeholders for particular business applications.
In the meantime, a wide variety of terminologies, motivations, approaches, and evaluation criteria have been developed within the research field of explainable artificial intelligence (XAI). With the amount of XAI methods vastly growing, a taxonomy of methods is needed by researchers as well as practitioners: To grasp the breadth of the topic, compare methods, and to select the right XAI method based on traits required by a specific use-case context. Many taxonomies for XAI methods of varying level of detail and depth can be found in the literature. While they often have a different focus, the…
Saarland University（萨尔大学）；Technical University Dortmund（多特蒙德工业大学）；Institute of Intelligent Software（智能软件研究所）
Previous research in Explainable Artificial Intelligence (XAI) suggests that a main aim of explainability approaches is to satisfy specific interests, goals, expectations, needs, and demands regarding artificial systems (we call these stakeholders' desiderata) in a variety of contexts. However, the literature on XAI is vast, spreads out across multiple largely disconnected disciplines, and it often remains unclear how explainability approaches are supposed to achieve the goal of satisfying stakeholders' desiderata. This paper discusses the main classes of stakeholders calling for explainabilit…
University of Twente（特文特大学）；University of Duisburg-Essen（杜伊斯堡-埃森大学）
The rising popularity of explainable artificial intelligence (XAI) to understand high-performing black boxes raised the question of how to evaluate explanations of machine learning (ML) models. While interpretability and explainability are often presented as a subjectively validated binary property, we consider it a multi-faceted concept. We identify 12 conceptual properties, such as Compactness and Correctness, that should be evaluated for comprehensively assessing the quality of an explanation. Our so-called Co-12 properties serve as categorization scheme for systematically reviewing the eva…
Departamento de Ciencias de la Computación e Inteligencia Artificial（计算机科学与人工智能系）；Datrik Intelligence, S.A.（Datrik智能股份公司）；Universidad de Sevilla（塞维利亚大学）
The aim of Predictive Maintenance, within the field of Prognostics and Health Management (PHM), is to identify and anticipate potential issues in the equipment before these become critical. The main challenge to be addressed is to assess the amount of time a piece of equipment will function effectively before it fails, which is known as Remaining Useful Life (RUL). Deep Learning (DL) models, such as Deep Convolutional Neural Networks (DCNN) and Long Short-Term Memory (LSTM) networks, have been widely adopted to address the task, with great success. However, it is well known that this kind of b…
Explainable AI for Safe and Trustworthy Autonomous Driving: A Systematic Review
School of Informatics, University of Edinburgh（爱丁堡大学信息学院）；Institute of Automotive Engineering, Technical University (TU) of Darmstadt（达姆施塔特工业大学汽车工程研究所）；School of Engineering and Physical Sciences, Heriot-Watt University（赫瑞瓦特大学工程与物理科学学院）
Brandon University（布兰登大学）；Pandit Deendayal Energy University（潘迪特·丁达亚尔能源大学）；Vellore Institute of Technology（韦洛尔理工学院）；Symbiosis International Deemed University（共生国际被视为大学）；The Open University（开放大学）
In the past few years, artificial intelligence (AI) techniques have been implemented in almost all verticals of human life. However, the results generated from the AI models often lag explainability. AI models often appear as a blackbox wherein developers are unable to explain or trace back the reasoning behind a specific decision. Explainable AI (XAI) is a rapid growing field of research which helps to extract information and also visualize the results generated with an optimum transparency. The present study provides and extensive review of the use of XAI in cybersecurity. Cybersecurity enab…
National Heart and Lung Institute, Imperial College London（帝国理工学院国家心肺研究所）；Royal Brompton Hospital（皇家布朗普顿医院）；Imperial Institute of Advanced Technology（帝国理工高等研究院）；Hangzhou Ocean’s Smart Boya Co., Ltd（杭州海洋智能博雅有限公司）；University of California, San Diego（加州大学圣地亚哥分校）；Shenzhen Second People’s Hospital（深圳市第二人民医院）
Explainable Artificial Intelligence (XAI) is an emerging research topic of machine learning aimed at unboxing how AI systems' black-box choices are made. This research field inspects the measures and models involved in decision-making and seeks solutions to explain them explicitly. Many of the machine learning algorithms can not manifest how and why a decision has been cast. This is particularly true of the most popular deep neural network approaches currently in use. Consequently, our confidence in AI systems can be hindered by the lack of explainability in these black-box models. The XAI bec…
Institute for Informatics and Telematics (IIT), National Research Council of Italy (CNR)（意大利国家研究委员会（CNR）信息学与电信研究所（IIT））
Nowadays Artificial Intelligence (AI) has become a fundamental component of healthcare applications, both clinical and remote, but the best performing AI systems are often too complex to be self-explaining. Explainable AI (XAI) techniques are defined to unveil the reasoning behind the system's predictions and decisions, and they become even more critical when dealing with sensitive and personal health data. It is worth noting that XAI has not gathered the same attention across different research areas and data types, especially in healthcare. In particular, many clinical and remote health appl…
Technische Universität Berlin（柏林工业大学）；Fraunhofer Heinrich Hertz Institute（弗劳恩霍夫海因里希·赫兹研究所）；BIFOLD – Berlin Institute for the Foundations of Learning and Data（BIFOLD——柏林学习与数据基础研究所）；Korea University（高丽大学）；Max Planck Institute for Informatics（马克斯·普朗克信息学研究所）；Google Research（谷歌研究院）
In addition to the impressive predictive power of machine learning (ML) models, more recently, explanation methods have emerged that enable an interpretation of complex non-linear learning models such as deep neural networks. Gaining a better understanding is especially important e.g. for safety-critical ML applications or medical diagnostics etc. While such Explainable AI (XAI) techniques have reached significant popularity for classifiers, so far little attention has been devoted to XAI for regression models (XAIR). In this review, we clarify the fundamental conceptual differences of XAI for…
Explainable Artificial Intelligence (XAI) in Computational Pathology: Definitions, Taxonomy, and Recommendations
Indiana University School of Medicine（印第安纳大学医学院）；Indiana University Melvin and Bren Simon Comprehensive Cancer Center（印第安纳大学梅尔文与布伦·西蒙综合癌症中心）；University of Warwick（华威大学）；Emory University（埃默里大学）；Radboud University Medical Center（拉德堡德大学医学中心）；Singapore General Hospital（新加坡中央医院）
Pioneer Alpha（先锋阿尔法）；Bangladesh University of Engineering and Technology（孟加拉工程技术大学）；University of Hertfordshire（赫特福德大学）；Westcliff university（韦斯特克利夫大学）；Clark Atlanta University（克拉克亚特兰大大学）
IBM Research AI（IBM研究院AI）；IBM Design（IBM设计）；IBM Research（IBM研究院）
A pervasive design issue of AI systems is their explainability--how to provide appropriate information to help users understand the AI. The technical field of explainable AI (XAI) has produced a rich toolbox of techniques. Designers are now tasked with the challenges of how to select the most suitable XAI techniques and translate them into UX solutions. Informed by our previous work studying design challenges around XAI UX, this work proposes a design process to tackle these challenges. We review our and related prior work to identify requirements that the process should fulfill, and according…
Interplay between Federated Learning and Explainable Artificial Intelligence: a Scoping Review
Simula Metropolitan Center for Digital Engineering（西米拉数字工程中心）；Karlsruhe Institute of Technology（卡尔斯鲁厄理工学院）；University Hospital Bonn（波恩大学医院）；Norwegian University of Science and Technology（挪威科技大学）；Charité - Universitätsmedizin Berlin（柏林夏里特医学院）；Birmingham City University（伯明翰城市大学）；ETH Zurich（苏黎世联邦理工学院）；Technical University of Munich（慕尼黑工业大学）
Evolutionary Computation and Explainable AI: A Roadmap to Understandable Intelligent Systems
Queen’s University（女王大学）；Newcastle University（纽卡斯尔大学）；University of Stirling（斯特灵大学）；University of Parma（帕尔马大学）；Robert Gordon University（罗伯特戈登大学）；University of Trento（特伦托大学）；Universiteit Leiden（莱顿大学）；University of Exeter（埃克塞特大学）
