Explainable AI (XAI) is essential for understanding machine learning (ML) decision-making and ensuring model trustworthiness in scientific applications. Prototype-based XAI methods offer an intrinsically interpretable alternative to post-hoc approaches which often yield inconsistent explanations. Prototype-based XAI methods make predictions based on the similarity between inputs and learned prototypes that represent typical characteristics of target classes. However, existing prototype-based models are primarily designed for standard RGB image data and are not optimized for the distinct, variable-specific channels commonly found in geoscientific image and raster datasets. In this study, we develop a prototype-based XAI approach tailored for multi-channel geospatial data, where each channel represents a distinct physical environmental variable or spectral channel. Our approach enables the model to identify separate, channel-specific prototypical characteristics sourced from multiple distinct training examples that inform how these features individually and in combination influence model prediction while achieving comparable performance to standard neural networks. We demonstrate this method through two geoscientific case studies: (1) classification of Madden Julian Oscillation phases using multi-variable climate data and (2) land-use classification from multispectral satellite imagery. This approach produces both local (instance-level) and global (model-level) explanations for providing insights into feature-relevance across channels. By explicitly incorporating channel-prototypes into the prediction process, we discuss how this approach enhances the transparency and trustworthiness of ML models for geoscientific learning tasks.
Graph Neural Networks (GNNs) have shown great ability in modeling graph-structured data for various domains. However, GNNs are known as black-box models that lack interpretability. Without understanding their inner working, we cannot fully trust them, which largely limits their adoption in high-stake scenarios. Though some initial efforts have been taken to interpret the predictions of GNNs, they mainly focus on providing post-hoc explanations using an additional explainer, which could misrepresent the true inner working mechanism of the target GNN. The works on self-explainable GNNs are rathe…
Diagnosing Alzheimer's disease (AD) involves a deliberate diagnostic process owing to its innate traits of irreversibility with subtle and gradual progression. These characteristics make AD biomarker identification from structural brain imaging (e.g., structural MRI) scans quite challenging. Furthermore, there is a high possibility of getting entangled with normal aging. We propose a novel deep-learning approach through eXplainable AD Likelihood Map Estimation (XADLiME) for AD progression modeling over 3D sMRIs using clinically-guided prototype learning. Specifically, we establish a set of top…
Uncertainty-Aware Explanations Through Probabilistic Self-Explainable Neural Networks
Department of Computer Science and Artificial Intelligence, University of the Basque Country UPV/EHU（计算机科学与人工智能系，巴斯克国家大学UPV/EHU）；Basque Center for Applied Mathematics (BCAM)（巴斯克应用数学中心）；Department of Computer Science, University of Oxford（计算机科学系，牛津大学）
Predefined Prototypes for Intra-Class Separation and Disentanglement
ViVoLab（维沃实验室）；Aragón Institute for Engineering Research (I3A)（阿拉贡工程研究所（I3A））；University of Zaragoza（萨拉戈萨大学）；LIUM（勒芒大学集成与多媒体实验室）；Le Mans University（勒芒大学）；LTCI（通信与集成实验室）；Télécom Paris（巴黎电信学院）；Institut Polytechnique de Paris（巴黎综合理工学院）
The University of Texas at Austin（德克萨斯大学奥斯汀分校）；Dell Medical School, The University of Texas at Austin（德克萨斯大学奥斯汀分校戴尔医学院）
The emergence of deep learning networks raises a need for explainable AI so that users and domain experts can be confident applying them to high-risk decisions. In this paper, we leverage data from the latent space induced by deep learning models to learn stereotypical representations or "prototypes" during training to elucidate the algorithmic decision-making process. We study how leveraging prototypes effect classification decisions of two dimensional time-series data in a few different settings: (1) electrocardiogram (ECG) waveforms to detect clinical bradycardia, a slowing of heart rate, i…
In this paper, we propose an elegant solution that is directly addressing the bottlenecks of the traditional deep learning approaches and offers a clearly explainable internal architecture that can outperform the existing methods, requires very little computational resources (no need for GPUs) and short training times (in the order of seconds). The proposed approach, xDNN is using prototypes. Prototypes are actual training data samples (images), which are local peaks of the empirical data distribution called typicality as well as of the data density. This generative model is identified in a cl…
From Black Box to Insight: Explainable AI for Extreme Event Preparedness
Department of Cybersecurity University at Albany, SUNY Albany, NY, USA；Dept. of Atmospheric \& Environmental Sciences University at Albany, SUNY Albany, NY, USA；Lally School of Management Rensselaer Polytechnic Institute Albany, NY, USA；Goddard Space Flight Center NASA Greenbelt, MD, USA
Prototype-Based Learning for Healthcare: A Demonstration of Interpretable AI
NEC Laboratories Europe GmbH（NEC欧洲实验室）；NEC Solution Innovators, Ltd.（NEC解决方案创新者有限公司）；Kurashiki Central Hospital Preventive Healthcare Plaza（Kurashiki中央医院预防保健广场）；Kurashiki Central Hospital（Kurashiki中央医院）
Graph-Structured Hyperdimensional Computing for Data-Efficient and Explainable Process-Structure-Property Prediction
School of Mechanical, Aerospace, and Manufacturing Engineering, University of Connecticut（康奈尔大学机械、航空航天与制造工程学院）；U.S. Army Combat Capabilities Development Command, Ground Vehicle Systems Center（美国陆军战斗能力发展指挥部地面车辆系统中心）
TimeXL: Explainable Multi-modal Time Series Prediction with LLM-in-the-Loop
School of Computing, University of Connecticut（大学计算机学院）；Data Science & System Security Department, NEC Labs America（数据科学与系统安全部，NEC美国实验室）；Kim Jaechul Graduate School of AI, KAIST（金 Jaechul人工智能研究生院，韩国科学技术院）
Algorithmic approaches to interpreting machine learning models have proliferated in recent years. We carry out human subject tests that are the first of their kind to isolate the effect of algorithmic explanations on a key aspect of model interpretability, simulatability, while avoiding important confounding experimental factors. A model is simulatable when a person can predict its behavior on new inputs. Through two kinds of simulation tests involving text and tabular data, we evaluate five explanations methods: (1) LIME, (2) Anchor, (3) Decision Boundary, (4) a Prototype model, and (5) a Com…
Recidivism prediction provides decision makers with an assessment of the likelihood that a criminal defendant will reoffend that can be used in pre-trial decision-making. It can also be used for prediction of locations where crimes most occur, profiles that are more likely to commit violent crimes. While such instruments are gaining increasing popularity, their use is controversial as they may present potential discriminatory bias in the risk assessment. In this paper we propose a new fair-by-design approach to predict recidivism. It is prototype-based, learns locally and extracts empirically …
RES-MR: Risk-Aware Reasoning for Explainable and Safe Medication Recommendation
Shandong Computer Science Center (National Supercomputer Center in Jinan)（山东省计算机科学中心（国家济南超级计算中心））；Qilu University of Technology (Shandong Academy of Sciences)（齐鲁工业大学（山东省科学院））；Shandong Fundamental Research Center for Computer Science（山东省计算机科学基础研究中心）；University of Technology Sydney（悉尼科技大学）
Technische Universität Kaiserslautern（凯泽斯劳滕工业大学）；German Research Center for Artificial Intelligence (DFKI)（德国人工智能研究中心（DFKI））
Deep learning methods have shown great success in several domains as they process a large amount of data efficiently, capable of solving complex classification, forecast, segmentation, and other tasks. However, they come with the inherent drawback of inexplicability limiting their applicability and trustworthiness. Although there exists work addressing this perspective, most of the existing approaches are limited to the image modality due to the intuitive and prominent concepts. Conversely, the concepts in the time-series domain are more complex and non-comprehensive but these and an explanati…
Comprehensive Evaluation of Prototype Neural Networks
Semantic Information Systems Group, Osnabrück University, Germany（语义信息系统组，奥斯纳布吕克大学，德国）；Research Department Cooperative and Autonomous Systems, German Research Center for Artificial Intelligence (DFKI), Germany（协作与自主系统研究部，德国人工智能研究中心（DFKI），德国）
APEX: Audio Prototype EXplanations for Classification Tasks
Department of Artificial Intelligence, Wroclaw University of Science and Technology, Poland（华沙理工大学人工智能系）；Resemble AI, USA（Resemble AI公司）；IDEAS Research Institute, Poland（波兰IDEAS研究院）；Faculty of Mathematics and Computer Science, Jagiellonian University, Poland（雅盖隆大学数学与计算机科学系）；Doctoral School of Exact and Natural Sciences, Jagiellonian University, Poland（雅盖隆大学博士学院）
From Confusion to Clarity: ProtoScore -- A Framework for Evaluating Prototype-Based XAI
Fraunhofer Institute for Manufacturing Engineering and Automation IPA（弗劳恩霍夫研究所（制造工程与自动化））；Institute of Industrial Manufacturing and Management IFF, University of Stuttgart（工业制造与管理研究所，斯图加特大学）；TUM School of Computation, Information and Technology, Technical University of Munich（慕尼黑技术大学计算、信息与技术学院）
Archetypal Graph Generative Models: Explainable and Identifiable Communities via Anchor-Dominant Convex Hulls
Yale Institute for Network Science, Yale University, USA（耶鲁网络科学研究所，耶鲁大学，美国）；Université Paris Saclay, Université Paris Cité, ENS Paris Saclay, CNRS, SSA, INSERM, Centre Borelli, France（巴黎萨克雷大学，巴黎城市大学，巴黎萨克雷高等学院，法国国家科学研究中心，SSA，INSERM，Borelli中心，法国）；École Polytechnique, Paris, France（巴黎高等理工学院，巴黎，法国）；Department of Informatics and Telecommunications, University of Peloponnese, Greece（信息与电信系，希腊皮埃利翁大学）
In this work, we propose the novel Prototypical Graph Regression Self-explainable Trees (ProGReST) model, which combines prototype learning, soft decision trees, and Graph Neural Networks. In contrast to other works, our model can be used to address various challenging tasks, including compound property prediction. In ProGReST, the rationale is obtained along with prediction due to the model's built-in interpretability. Additionally, we introduce a new graph prototype projection to accelerate model training. Finally, we evaluate PRoGReST on a wide range of chemical datasets for molecular prope…
School of Computing and Communications, Lancaster University（兰卡斯特大学计算与通信学院）
In this paper we introduce the DMR -- a prototype-based method and network architecture for deep learning which is using a decision tree (DT)-based inference and synthetic data to balance the classes. It builds upon the recently introduced xDNN method addressing more complex multi-class problems, specifically when classes are highly imbalanced. DMR moves away from a direct decision based on all classes towards a layered DT of pair-wise class comparisons. In addition, it forces the prototypes to be balanced between classes regardless of possible class imbalances of the training data. It has two…
Machine Learning has become an integral part of engineering design and decision making in several domains, including sports. Deep Neural Networks (DNNs) have been the state-of-the-art methods for predicting outcomes of professional sports events. However, apart from getting highly accurate predictions on these sports events outcomes, it is necessary to answer questions such as "Why did the model predict that Team A would win Match X against Team B?" DNNs are inherently black-box in nature. Therefore, it is required to provide high-quality interpretable, and understandable explanations for a mo…
University of Bucharest（布加勒斯特大学）；University Politehnica of Bucharest（布加勒斯特理工大学）；Sparktech Software
With the growing complexity of deep learning methods adopted in practical applications, there is an increasing and stringent need to explain and interpret the decisions of such methods. In this work, we focus on explainable AI and propose a novel generic and model-agnostic framework for synthesizing input exemplars that maximize a desired response from a machine learning model. To this end, we use a generative model, which acts as a prior for generating data, and traverse its latent space using a novel evolutionary strategy with momentum updates. Our framework is generic because (i) it can emp…
TCR-EML: Explainable Model Layers for TCR-pMHC Prediction
Department of Computer Science, Tulane University（计算机科学系，路易斯安那大学）；Department of Biochemistry and Molecular Biology, Tulane University School of Medicine（生物化学与分子生物学系，路易斯安那大学医学学院）
Why Does It Look There? Structured Explanations for Image Classification
Department of Computer Science, Tulane University, New Orleans LA 70118, USA（计算机科学系，路易斯安那大学，新奥尔良 LA 70118, 美国）；Department of Biochemistry and Molecular Biology, Tulane University School of Medicine, New Orleans LA 70112, USA（生物化学与分子生物学系，路易斯安那大学医学院，新奥尔良 LA 70112, 美国）
