Deep neural networks (DNNs) have become a proven and indispensable machine learning tool. As a black-box model, it remains difficult to diagnose what aspects of the model's input drive the decisions of a DNN. In countless real-world domains, from legislation and law enforcement to healthcare, such diagnosis is essential to ensure that DNN decisions are driven by aspects appropriate in the context of its use. The development of methods and studies enabling the explanation of a DNN's decisions has thus blossomed into an active, broad area of research. A practitioner wanting to study explainable deep learning may be intimidated by the plethora of orthogonal directions the field has taken. This complexity is further exacerbated by competing definitions of what it means ``to explain'' the actions of a DNN and to evaluate an approach's ``ability to explain''. This article offers a field guide to explore the space of explainable deep learning aimed at those uninitiated in the field. The field guide: i) Introduces three simple dimensions defining the space of foundational methods that contribute to explainable deep learning, ii) discusses the evaluations for model explanations, iii) places explainability in the context of other related deep learning research areas, and iv) finally elaborates on user-oriented explanation designing and potential future directions on explainable deep learning. We hope the guide is used as an easy-to-digest starting point for those just embarking on research in this field.
In current deep network architectures, deeper layers in networks tend to contain hundreds of independent neurons which makes it hard for humans to understand how they interact with each other. By organizing the neurons by correlation, humans can observe how clusters of neighbouring neurons interact with each other. In this paper, we propose a novel algorithm for back propagation, called Locality Guided Neural Network(LGNN) for training networks that preserves locality between neighbouring neurons within each layer of a deep network. Heavily motivated by Self-Organizing Map (SOM), the goal is t…
A Verifier-Guided Explainable Reasoning Framework with Gold-Anchored QLoRA, Task-Aware Mixture-of-Experts, and Group-Relative RLVR
University of Information Technology (UIT)（信息科技大学）；Ho Chi Minh City University of Technology (HCMUT)（胡志明市理工大学）；Vietnam National University, Ho Chi Minh City（越南国家大学胡志明市分校）；University of Economics Ho Chi Minh City (UEH)（胡志明市经济大学）
GALAX: Graph-Augmented Language Model for Explainable Reinforcement-Guided Subgraph Reasoning in Precision Medicine
I2DB, Washington University School of Medicine（I2DB，华盛顿大学医学学院）；Department of Computer Science and Engineering, Washington University in St. Louis（计算机科学与工程系，华盛顿大学圣路易斯分校）；Department of Genetics, Washington University School of Medicine（遗传学系，华盛顿大学医学学院）
Simon Fraser University（西蒙弗雷泽大学）；The University of British Columbia（不列颠哥伦比亚大学）
Explainable artificial intelligence (XAI) is essential for enabling clinical users to get informed decision support from AI and comply with evidence-based medical practice. Applying XAI in clinical settings requires proper evaluation criteria to ensure the explanation technique is both technically sound and clinically useful, but specific support is lacking to achieve this goal. To bridge the research gap, we propose the Clinical XAI Guidelines that consist of five criteria a clinical XAI needs to be optimized for. The guidelines recommend choosing an explanation form based on Guideline 1 (G1)…
Multimodal Doctor-in-the-Loop: A Clinically-Guided Explainable Framework for Predicting Pathological Response in Non-Small Cell Lung Cancer
Research Unit of Computer Systems and Bioinformatics, Department of Engineering（计算机系统与生物信息学研究单位，工程系）；Research Unit of Radiation Oncology, Department of Medicine and Surgery（放射肿瘤学研究单位，医学与外科系）；Research Unit of Anatomical Pathology, Department of Medicine and Surgery（病理学研究单位，医学与外科系）
Artificial Intelligence for CRISPR Guide RNA Design: Explainable Models and Off-Target Safety
Department of Computer Engineering, Ma.C., Islamic Azad University（计算机工程系，伊斯兰阿兹德大学）；Department of Biological Sciences and Technologies, Faculty of Basic Sciences, Islamic Azad University（基础科学学院生物科学与技术系，伊斯兰阿兹德大学）
University of Alberta（阿尔伯塔大学）；Alberta Machine Intelligence Institute (AMII)（阿尔伯塔机器智能研究所）
We focus on exploring various approaches of Zero-Shot Learning (ZSL) and their explainability for a challenging yet important supervised learning task notorious for training data scarcity, i.e. Depression Symptoms Detection (DSD) from text. We start with a comprehensive synthesis of different components of our ZSL modeling and analysis of our ground truth samples and Depression symptom clues curation process with the help of a practicing clinician. We next analyze the accuracy of various state-of-the-art ZSL models and their potential enhancements for our task. Further, we sketch a framework f…
ENSTA, Institut Polytechnique Paris（巴黎综合理工学院ENSTA）；INRIA（法国国家信息与自动化研究所）；Segula Technologies（赛科集团）；Sorbonne Université（索邦大学）；Free University of Bozen-Bolzano（波尔扎诺自由大学）；Tinubu Square（Tinubu Square公司）；Fondazione Bruno Kessler（布鲁诺·凯斯勒基金会）；City University London（伦敦城市大学）；Medical University Graz（格拉茨医科大学）；INESC-ID（葡萄牙信息与通信技术研究院）；Instituto Superior Técnico（里斯本高等理工学院）；Institut Supérieur d’Électronique de Paris (ISEP)（巴黎高等电子学院）；Alberta Machine Intelligence Institute（阿尔伯塔机器智能研究所）
Last years have been characterized by an upsurge of opaque automatic decision support systems, such as Deep Neural Networks (DNNs). Although they have great generalization and prediction skills, their functioning does not allow obtaining detailed explanations of their behaviour. As opaque machine learning models are increasingly being employed to make important predictions in critical environments, the danger is to create and use decisions that are not justifiable or legitimate. Therefore, there is a general agreement on the importance of endowing machine learning models with explainability. E…
REMEMBER: Retrieval-based Explainable Multimodal Evidence-guided Modeling for Brain Evaluation and Reasoning in Zero- and Few-shot Neurodegenerative Diagnosis
Lausanne University Hospital (CHUV)（洛桑大学医院（CHUV））；University of Lausanne (UNIL)（洛桑大学（UNIL））；Vietnam National University（越南国立大学）；University of Science, Vietnam National University Ho Chi Minh City（越南国立大学胡志明市科学大学）；International University, Vietnam National University Ho Chi Minh City（越南国立大学胡志明市国际大学）
A plethora of recent research has proposed several automated methods based on machine learning (ML) and deep learning (DL) to detect cybersickness in Virtual reality (VR). However, these detection methods are perceived as computationally intensive and black-box methods. Thus, those techniques are neither trustworthy nor practical for deploying on standalone VR head-mounted displays (HMDs). This work presents an explainable artificial intelligence (XAI)-based framework VR-LENS for developing cybersickness detection ML models, explaining them, reducing their size, and deploying them in a Qualcom…
A Comprehensive Guide to Explainable AI: From Classical Models to LLMs
National Tsing Hua University（国立清华大学）；Indiana University（印第安纳大学）；National University of Singapore（新加坡国立大学）；Kyoto University（京都大学）；AppCubic；Rutgers University（罗格斯大学）；University of Wisconsin-Madison（威斯康星大学麦迪逊分校）；Purdue University（普渡大学）；Georgia Institute of Technology（佐治亚理工学院）；National Taiwan Normal University（国立台湾师范大学）；University of Hawaii（夏威夷大学）；Xi’an Jiaotong-Liverpool University（西交利物浦大学）；Pingtan Research Institute of Xiamen University（厦门大学平潭研究院）；University of Manchester（曼彻斯特大学）；Aarhus University（奥胡斯大学）；Imperial College London（伦敦帝国学院）；School of Visual Arts（视觉艺术学院）；The University of Texas at Dallas（德克萨斯大学达拉斯分校）
Finding the right XAI method -- A Guide for the Evaluation and Ranking of Explainable AI Methods in Climate Science
Technische Universität Berlin（柏林工业大学）；ATB（莱布尼茨农业工程与生物经济研究所）；University of Leipzig（莱比锡大学）；University of Reading（雷丁大学）；BIFOLD – Berlin Institute for the Foundations of Learning and Data（柏林学习与数据基础研究所（BIFOLD））；UiT the Arctic University of Norway（挪威北极大学）；University of Potsdam（波茨坦大学）
BACON: A fully explainable AI model with graded logic for decision making problems
Department of Information Systems University of Maryland, Baltimore County (UMBC)（信息系统系大学马里兰大学巴尔的摩县（UMBC））；Department of Computer Science San Francisco State University（计算机科学系旧金山州立大学）；Department of Information Systems UMBC（信息系统系大学马里兰大学巴尔的摩县（UMBC））
Center for Artificial Intelligence Research（人工智能研究中心）；University of Agder（阿格德大学）
The past decade has seen significant progress in artificial intelligence (AI), which has resulted in algorithms being adopted for resolving a variety of problems. However, this success has been met by increasing model complexity and employing black-box AI models that lack transparency. In response to this need, Explainable AI (XAI) has been proposed to make AI more transparent and thus advance the adoption of AI in critical domains. Although there are several reviews of XAI topics in the literature that identified challenges and potential research directions in XAI, these challenges and resear…
A Survey on Human-Centered Evaluation of Explainable AI Methods in Clinical Decision Support Systems
Nova School of Science and Technology, Universidade NOVA de Lisboa（诺瓦科学与技术学院，诺瓦里斯本大学）；Nova School of Business and Economics, Universidade NOVA de Lisboa（诺瓦商学院与经济学院，诺瓦里斯本大学）；Carnegie Mellon University（卡内基梅隆大学）
ReLMXEL: Adaptive RL-Based Memory Controller with Explainable Energy and Latency Optimization
Department of Mathematics and Computer Science（数学与计算机科学系）；Sri Sathya Sai Institute of Higher Learning（Sri Sathya Sai高等研究院）；Department of Computer Science and Engineering（计算机科学与工程系）；Indian Institute of Technology Ropar（印度理工学院罗帕尔分校）；AI Performance Engineer（AI性能工程师）；Red Hat（红帽公司）
Dalhousie University（达尔豪斯大学）；Universidade de São Paulo（圣保罗大学）
As neural networks become the tool of choice to solve an increasing variety of problems in our society, adversarial attacks become critical. The possibility of generating data instances deliberately designed to fool a network's analysis can have disastrous consequences. Recent work has shown that commonly used methods for model training often result in fragile abstract representations that are particularly vulnerable to such attacks. This paper presents a visual framework to investigate neural network models subjected to adversarial examples, revealing how models' perception of the adversarial…
Meta Reality Labs（Meta现实实验室）；Purdue（普渡大学）；Aarhus Univ（奥胡斯大学）；Dartmouth（达特茅斯学院）；Cambridge（剑桥大学）
Explainable AI (XAI) has established itself as an important component of AI-driven interactive systems. With Augmented Reality (AR) becoming more integrated in daily lives, the role of XAI also becomes essential in AR because end-users will frequently interact with intelligent services. However, it is unclear how to design effective XAI experiences for AR. We propose XAIR, a design framework that addresses "when", "what", and "how" to provide explanations of AI output in AR. The framework was based on a multi-disciplinary literature review of XAI and HCI research, a large-scale survey probing …
An ExplainableFair Framework for Prediction of Substance Use Disorder Treatment Completion
1 College of Computing；Informatics Drexel University, Philadelphia PA, USA Email: mml367,xw388,cc3859；2 Butler Center for Research Hazelden Betty Ford Foundation, Minnesota, USA Email: jbraughton
Technical University of Denmark（丹麦技术大学）；Omhu A/S（Omhu公司）；University of Copenhagen（哥本哈根大学）；Rigshospitalet, Copenhagen University Hospital（哥本哈根大学医院国立医院）
Dermatological diagnosis automation is essential in addressing the high prevalence of skin diseases and critical shortage of dermatologists. Despite approaching expert-level diagnosis performance, convolutional neural network (ConvNet) adoption in clinical practice is impeded by their limited explainability, and by subjective, expensive explainability validations. We introduce DermX and DermX+, an end-to-end framework for explainable automated dermatological diagnosis. DermX is a clinically-inspired explainable dermatological diagnosis ConvNet, trained using DermXDB, a 554 image dataset annota…
Explainable Deep Learning-based Classification of Wolff-Parkinson-White Electrocardiographic Signals
Department of Mechanical Engineering, University of Washington（华盛顿大学机械工程系）；Division of Biophysics and Medical Physics, Gottfried Schatz Research Center, Medical University of Graz（格拉茨医科大学生物物理与医学物理系）；Scientific Computing and Imaging Institute, University of Utah（犹他大学科学计算与成像研究所）；Department of Biomedical Engineering, University of Utah（犹他大学生物医学工程系）
