Shruthi Chari, Daniel M. Gruen, Oshani Seneviratne, Deborah L. McGuinness
备注：
S. Chari, D. M. Gruen, O. Seneviratne, D. L. McGuinness, "Directions for Explainable Knowledge-Enabled Systems". In: Ilaria Tiddi, Freddy Lecue, Pascal Hitzler (eds.), Knowledge Graphs for eXplainable AI -- Foundations, Applications and Challenges. Studies on the Semantic Web, IOS Press, Amsterdam, 2020, to appear
Interest in the field of Explainable Artificial Intelligence has been growing for decades and has accelerated recently. As Artificial Intelligence models have become more complex, and often more opaque, with the incorporation of complex machine learning techniques, explainability has become more critical. Recently, researchers have been investigating and tackling explainability with a user-centric focus, looking for explanations to consider trustworthiness, comprehensibility, explicit provenance, and context-awareness. In this chapter, we leverage our survey of explanation literature in Artificial Intelligence and closely related fields and use these past efforts to generate a set of explanation types that we feel reflect the expanded needs of explanation for today's artificial intelligence applications. We define each type and provide an example question that would motivate the need for this style of explanation. We believe this set of explanation types will help future system designers in their generation and prioritization of requirements and further help generate explanations that are better aligned to users' and situational needs.
Explainability has been an important goal since the early days of Artificial Intelligence. Several approaches for producing explanations have been developed. However, many of these approaches were tightly coupled with the capabilities of the artificial intelligence systems at the time. With the proliferation of AI-enabled systems in sometimes critical settings, there is a need for them to be explainable to end-users and decision-makers. We present a historical overview of explainable artificial intelligence systems, with a focus on knowledge-enabled systems, spanning the expert systems, cognit…
Modern data analytics underpinned by machine learning techniques has become a key enabler to the automation of data-led decision making. As an important branch of state-of-the-art data analytics, business process predictions are also faced with a challenge in regard to the lack of explanation to the reasoning and decision by the underlying `black-box' prediction models. With the development of interpretable machine learning techniques, explanations can be generated for a black-box model, making it possible for (human) users to access the reasoning behind machine learned predictions. In this pa…
Artificial intelligence (AI) enables machines to learn from human experience, adjust to new inputs, and perform human-like tasks. AI is progressing rapidly and is transforming the way businesses operate, from process automation to cognitive augmentation of tasks and intelligent process/data analytics. However, the main challenge for human users would be to understand and appropriately trust the result of AI algorithms and methods. In this paper, to address this challenge, we study and analyze the recent work done in Explainable Artificial Intelligence (XAI) methods and tools. We introduce a no…
Centre for Forensic Science, University of Technology Sydney（悉尼科技大学法医科学中心）
Explainable Artificial Intelligence (XAI) is an emerging area of research in the field of Artificial Intelligence (AI). XAI can explain how AI obtained a particular solution (e.g., classification or object detection) and can also answer other "wh" questions. This explainability is not possible in traditional AI. Explainability is essential for critical applications, such as defense, health care, law and order, and autonomous driving vehicles, etc, where the know-how is required for trust and transparency. A number of XAI techniques so far have been purposed for such applications. This paper pr…
Explainability is highly-desired in Machine Learning (ML) systems supporting high-stakes policy decisions in areas such as health, criminal justice, education, and employment. While the field of explainable ML has expanded in recent years, much of this work has not taken real-world needs into account. A majority of proposed methods are designed with \textit{generic} explainability goals without well-defined use-cases or intended end-users and evaluated on simplified tasks, benchmark problems/datasets, or with proxy users (e.g., AMT). We argue that these simplified evaluation settings do not ca…
Center for Artificial Intelligence Research（人工智能研究中心）；University of Agder（阿格德大学）
The past decade has seen significant progress in artificial intelligence (AI), which has resulted in algorithms being adopted for resolving a variety of problems. However, this success has been met by increasing model complexity and employing black-box AI models that lack transparency. In response to this need, Explainable AI (XAI) has been proposed to make AI more transparent and thus advance the adoption of AI in critical domains. Although there are several reviews of XAI topics in the literature that identified challenges and potential research directions in XAI, these challenges and resear…
In this work, we report the practical and theoretical aspects of Explainable AI (XAI) identified in some fundamental literature. Although there is a vast body of work on representing the XAI backgrounds, most of the corpuses pinpoint a discrete direction of thoughts. Providing insights into literature in practice and theory concurrently is still a gap in this field. This is important as such connection facilitates a learning process for the early stage XAI researchers and give a bright stand for the experienced XAI scholars. Respectively, we first focus on the categories of black-box explanati…
Mississippi State University（密西西比州立大学）；U.S Army Engineer Research and Development Center（美国陆军工程研究与发展中心）
Modern Artificial Intelligence (AI) enabled Intrusion Detection Systems (IDS) are complex black boxes. This means that a security analyst will have little to no explanation or clarification on why an IDS model made a particular prediction. A potential solution to this problem is to research and develop Explainable Intrusion Detection Systems (X-IDS) based on current capabilities in Explainable Artificial Intelligence (XAI). In this paper, we create a Self Organizing Maps (SOMs) based X-IDS system that is capable of producing explanatory visualizations. We leverage SOM's explainability to creat…
Fraunhofer Institute for Integrated Circuits IIS（弗劳恩霍夫集成电路研究所IIS）
After the tremendous advances of deep learning and other AI methods, more attention is flowing into other properties of modern approaches, such as interpretability, fairness, etc. combined in frameworks like Responsible AI. Two research directions, namely Explainable AI and Uncertainty Quantification are becoming more and more important, but have been so far never combined and jointly explored. In this paper, I show how both research areas provide potential for combination, why more research should be done in this direction and how this would lead to an increase in trustability in AI systems.
University of Washington（华盛顿大学）；Arthur AI；BITS Pilani, Goa Campus（贝拉理工学院果阿校区）；University of Maryland（马里兰大学）
As machine learning (ML) systems take a more prominent and central role in contributing to life-impacting decisions, ensuring their trustworthiness and accountability is of utmost importance. Explanations sit at the core of these desirable attributes of a ML system. The emerging field is frequently called ``Explainable AI (XAI)'' or ``Explainable ML.'' The goal of explainable ML is to intuitively explain the predictions of a ML system, while adhering to the needs to various stakeholders. Many explanation techniques were developed with contributions from both academia and industry. However, the…
Mepco Schlenk Engineering College（梅普科施伦克工程学院）；Pusan National University（釜山国立大学）；Shenzhen University（深圳大学）；Zhejiang University（浙江大学）；Zhejiang Lab（之江实验室）；Southeast University（东南大学）
Explainable Artificial Intelligence (XAI) is transforming the field of Artificial Intelligence (AI) by enhancing the trust of end-users in machines. As the number of connected devices keeps on growing, the Internet of Things (IoT) market needs to be trustworthy for the end-users. However, existing literature still lacks a systematic and comprehensive survey work on the use of XAI for IoT. To bridge this lacking, in this paper, we address the XAI frameworks with a focus on their characteristics and support for IoT. We illustrate the widely-used XAI services for IoT applications, such as securit…
School of Computer Science, College of Science and Health, Technological University Dublin（都柏林理工大学科学与健康学院计算机科学学院）；Technological University Dublin（都柏林理工大学）
Explainable Artificial Intelligence (XAI) has experienced a significant growth over the last few years. This is due to the widespread application of machine learning, particularly deep learning, that has led to the development of highly accurate models but lack explainability and interpretability. A plethora of methods to tackle this problem have been proposed, developed and tested. This systematic review contributes to the body of knowledge by clustering these methods with a hierarchical classification system with four main clusters: review articles, theories and notions, methods and their ev…
Explainable Artificial Intelligence: A Survey of Needs, Techniques, Applications, and Future Direction
University of Gondar（贡德尔大学）；College of Engineering and Applied Science, University of Colorado, Colorado Springs（科罗拉多大学科罗拉多斯普林斯分校工程与应用科学学院）；Can Tho University（芹苴大学）；Ewha Womans University（梨花女子大学）；University of Colorado, Colorado Springs（科罗拉多大学科罗拉多斯普林斯分校）
Mississippi State University（密西西比州立大学）；U.S. Army Engineer Research and Development Center（美国陆军工程师研发中心）
The current state of the art systems in Artificial Intelligence (AI) enabled intrusion detection use a variety of black box methods. These black box methods are generally trained using Error Based Learning (EBL) techniques with a focus on creating accurate models. These models have high performative costs and are not easily explainable. A white box Competitive Learning (CL) based eXplainable Intrusion Detection System (X-IDS) offers a potential solution to these problem. CL models utilize an entirely different learning paradigm than EBL approaches. This different learning process makes the CL …
Can Requirements Engineering Support Explainable Artificial Intelligence? Towards a User-Centric Approach for Explainability Requirements
Institute of Software Engineering University of Stuttgart Stuttgart, Germany Email；Institute of Software Engineering University of Stuttgart Stuttgart, Germany
Donders Institute for Brain, Cognition and Behaviour（唐德斯大脑、认知与行为研究所）；Radboud University Nijmegen（拉德堡德大学奈梅亨分校）；Amazon（亚马逊）；Tenet3, LLC（Tenet3有限责任公司）
Deep neural networks (DNNs) have become a proven and indispensable machine learning tool. As a black-box model, it remains difficult to diagnose what aspects of the model's input drive the decisions of a DNN. In countless real-world domains, from legislation and law enforcement to healthcare, such diagnosis is essential to ensure that DNN decisions are driven by aspects appropriate in the context of its use. The development of methods and studies enabling the explanation of a DNN's decisions has thus blossomed into an active, broad area of research. A practitioner wanting to study explainable …
Queensland University of Technology（昆士兰科技大学）；INESC-ID Lisboa（里斯本系统与计算机工程研究所）；Instituto Superior Técnico（高等技术学院）；ULisboa（里斯本大学）
There has been a growing interest in model-agnostic methods that can make deep learning models more transparent and explainable to a user. Some researchers recently argued that for a machine to achieve a certain degree of human-level explainability, this machine needs to provide human causally understandable explanations, also known as causability. A specific class of algorithms that have the potential to provide causability are counterfactuals. This paper presents an in-depth systematic review of the diverse existing body of literature on counterfactuals and causability for explainable artifi…
One of the core envisions of the sixth-generation (6G) wireless networks is to accumulate artificial intelligence (AI) for autonomous controlling of the Internet of Everything (IoE). Particularly, the quality of IoE services delivery must be maintained by analyzing contextual metrics of IoE such as people, data, process, and things. However, the challenges incorporate when the AI model conceives a lake of interpretation and intuition to the network service provider. Therefore, this paper provides an explainable artificial intelligence (XAI) framework for quality-aware IoE service delivery that…
University of Hartford（哈特福德大学）；Tennessee Tech University（田纳西理工大学）；Edith Cowan University（伊迪丝·考恩大学）
The lack of explainability of a decision from an Artificial Intelligence (AI) based "black box" system/model, despite its superiority in many real-world applications, is a key stumbling block for adopting AI in many high stakes applications of different domain or industry. While many popular Explainable Artificial Intelligence (XAI) methods or approaches are available to facilitate a human-friendly explanation of the decision, each has its own merits and demerits, with a plethora of open challenges. We demonstrate popular XAI methods with a mutual case study/task (i.e., credit default predicti…
Explainable Artificial Intelligence (XAI) 2.0: A Manifesto of Open Challenges and Interdisciplinary Research Directions
Technological University Dublin（都柏林理工大学）；University of Zagreb（萨格勒布大学）；University of Milano-Bicocca（米兰-比科卡大学）；IRCCS Ospedale Galeazzi Sant’Ambrogio（IRCCS 加莱阿齐-圣安布罗焦医院）；Korea Advanced Institute of Science & Technology（韩国科学技术院）；INEEJI Corporation（INEEJI公司）；University of Padua（帕多瓦大学）；TECNALIA（泰克纳利亚技术中心）；University of the Basque Country (UPV/EHU)（巴斯克大学）；University of Pisa（比萨大学）；Meiji University（明治大学）；University of Granada（格拉纳达大学）；University of Natural Resources and Life Sciences Vienna（维也纳自然资源与生命科学大学）；Lancaster University（兰卡斯特大学）；The University of Queensland（昆士兰大学）
Research Center for Information Technology Innovation, Academia Sinica（中央研究院資訊科技創新研究中心）；Chang Gung University（長庚大學）
In countries that enabled patients to choose their own providers, a common problem is that the patients did not make rational decisions, and hence, fail to use healthcare resources efficiently. This might cause problems such as overwhelming tertiary facilities with mild condition patients, thus limiting their capacity of treating acute and critical patients. To address such maldistributed patient volume, it is essential to oversee patients choices before further evaluation of a policy or resource allocation. This study used nationwide insurance data, accumulated possible features discussed in …
Explainable recommendation attempts to develop models that generate not only high-quality recommendations but also intuitive explanations. The explanations may either be post-hoc or directly come from an explainable model (also called interpretable or transparent model in some contexts). Explainable recommendation tries to address the problem of why: by providing explanations to users or system designers, it helps humans to understand why certain items are recommended by the algorithm, where the human can either be users or system designers. Explainable recommendation helps to improve the tran…
Explainable AI-based Intrusion Detection System for Industry 5.0: An Overview of the Literature, associated Challenges, the existing Solutions, and Potential Research Directions
Hamad bin Khalifa University（哈马德·本·哈利法大学）；Qatar Computing Research Institute（卡塔尔计算研究所）；Munster Technological University（芒斯特理工大学）；Toronto Metropolitan University（多伦多都会大学）
A Comprehensive Review on Financial Explainable AI
Nanyang Technological University（南洋理工大学）；National University of Singapore（新加坡国立大学）；Asian Institute of Digital Finance at NUS（新加坡国立大学亚洲数字金融研究所）；Imperial College London（帝国理工学院）
Unveiling Black-boxes: Explainable Deep Learning Models for Patent Classification
Fraunhofer-Institute for Applied Information Technology FIT（弗劳恩霍夫应用信息技术研究所）；University of Siegen（锡根大学）；RWTH Aachen University（亚琛工业大学）；Bonn-Rhein-Sieg University of Applied Sciences（波恩-莱茵-锡格应用科学大学）
On the explainable properties of 1-Lipschitz Neural Networks: An Optimal Transport Perspective
Université Paul-Sabatier（保罗萨巴蒂尔大学）；Institut de Recherche Technologique Saint-Exupéry（圣埃克苏佩里技术研究院）；Carney Institute for Brain Science,Brown University（布朗大学卡尼脑科学研究所）；Innovation & Research Division, SNCF（法国国家铁路公司创新与研究部）
Mind the Gap! Bridging Explainable Artificial Intelligence and Human Understanding with Luhmann's Functional Theory of Communication
School of Law, Birkbeck, University of London（伦敦大学伯贝克学院法学院）；Department of Computer Science, ETH Zurich（苏黎世联邦理工学院计算机科学系）；ARC Centre of Excellence for Automated Decision-Making and Society, RMIT University（皇家墨尔本理工大学自动化决策与社会ARC卓越研究中心）
Nowadays, deep neural networks are widely used in mission critical systems such as healthcare, self-driving vehicles, and military which have direct impact on human lives. However, the black-box nature of deep neural networks challenges its use in mission critical applications, raising ethical and judicial concerns inducing lack of trust. Explainable Artificial Intelligence (XAI) is a field of Artificial Intelligence (AI) that promotes a set of tools, techniques, and algorithms that can generate high-quality interpretable, intuitive, human-understandable explanations of AI decisions. In additi…
School of Business Administration, Southwestern University of Finance and Economics（西南财经大学工商管理学院）；Institute of Systems Engineering, Dalian University of Technology（大连理工大学系统工程研究所）；Business School, Hohai University（河海大学商学院）；Business School, Sichuan University（四川大学商学院）；School of Economics and Management, Southwest Jiaotong University（西南交通大学经济管理学院）；School of Information Management and Statistics, Hubei University of Economics（湖北经济学院信息管理与统计学院）；Andalusian Research Institute on Data Science and Computational Intelligence (DaSCI), University of Granada（格拉纳达大学安达卢西亚数据科学与计算智能研究所）；Faculty of Computing and Information Technology, King Abdulaziz University（阿卜杜勒阿齐兹大学计算机与信息技术学院）
Distributed linguistic representations are powerful tools for modelling the uncertainty and complexity of preference information in linguistic decision making. To provide a comprehensive perspective on the development of distributed linguistic representations in decision making, we present the taxonomy of existing distributed linguistic representations. Then, we review the key elements of distributed linguistic information processing in decision making, including the distance measurement, aggregation methods, distributed linguistic preference relations, and distributed linguistic multiple attr…
A number of exciting advances have been made in automated fact-checking thanks to increasingly larger datasets and more powerful systems, leading to improvements in the complexity of claims which can be accurately fact-checked. However, despite these advances, there are still desirable functionalities missing from the fact-checking pipeline. In this survey, we focus on the explanation functionality -- that is fact-checking systems providing reasons for their predictions. We summarize existing methods for explaining the predictions of fact-checking systems and we explore trends in this topic. F…
Are Rationales Necessary and Sufficient? Tuning LLMs for Explainable Misinformation Detection
College of Computer Science and Technology, Jilin University（吉林大学计算机科学与技术学院）；Tongyi Lab, Alibaba Group（阿里云实验室）；School of Artificial Intelligence, Jilin University（吉林大学人工智能学院）；College of Computer Science and Technology, Zhejiang University（浙江大学计算机科学与技术学院）
University of Sindh（信德大学）；Institute of Business Administration, University of Sindh（信德大学工商管理学院）；Nimbus Research Centre（尼姆布斯研究中心）；Munster Technological University（芒斯特理工大学）
After affecting the world in unexpected ways, COVID-19 has started mutating which is evident with the insurgence of its new variants. The governments, hospitals, schools, industries, and humans, in general, are looking for a potential solution in the vaccine which will eventually be available but its timeline for eradicating the virus is yet unknown. Several researchers have encouraged and recommended the use of good practices such as physical healthcare monitoring, immunity-boosting, personal hygiene, mental healthcare, and contact tracing for slowing down the spread of the virus. In this art…
A Survey on Explainable Artificial Intelligence for Cybersecurity
Princess Sumaya University for Technology（苏玛雅公主科技大学）；Concordia University（康考迪亚大学）；Polytechnique Montréal（蒙特利尔理工学院）；Khalifa University（哈利法大学）；University of Waterloo（滑铁卢大学）；Lebanese American University（黎巴嫩美国大学）；New York University（纽约大学）
