From Prototypes to Sparse ECG Explanations: SHAP-Driven Counterfactuals for Multivariate Time-Series Multi-class Classification
Jagiellonian Human-Centered AI Lab, Mark Kac Center for Complex Systems Research（杰哥利昂人中心人工智能实验室、马克·卡复杂系统研究中心）；Jagiellonian University（杰哥利昂大学）；Department of Computer Science（计算机科学系）；Norwegian University of Science and Technology (NTNU)（挪威科学技术大学（NTNU））；Department of Human-Centered Artificial Intelligence（人中心人工智能系）；Institute of Applied Computer Science（应用计算机科学研究所）；School of Information Technology（信息科技学院）
Maciej Mozolewski, Betül Bayrak, Kerstin Bach, Grzegorz J. Nalepa
In eXplainable Artificial Intelligence (XAI), instance-based explanations for time series have gained increasing attention due to their potential for actionable and interpretable insights in domains such as healthcare. Addressing the challenges of explainability of state-of-the-art models, we propose a prototype-driven framework for generating sparse counterfactual explanations tailored to 12-lead ECG classification models. Our method employs SHAP-based thresholds to identify critical signal segments and convert them into interval rules, uses Dynamic Time Warping (DTW) and medoid clustering to extract representative prototypes, and aligns these prototypes to query R-peaks for coherence with the sample being explained. The framework generates counterfactuals that modify only 78% of the original signal while maintaining 81.3% validity across all classes and achieving 43% improvement in temporal stability. We evaluate three variants of our approach, Original, Sparse, and Aligned Sparse, with class-specific performance ranging from 98.9% validity for myocardial infarction (MI) to challenges with hypertrophy (HYP) detection (13.2%). This approach supports near realtime generation (< 1 second) of clinically valid counterfactuals and provides a foundation for interactive explanation platforms. Our findings establish design principles for physiologically-aware counterfactual explanations in AI-based diagnosis systems and outline pathways toward user-controlled explanation interfaces for clinical deployment.
With the advent of GDPR, the domain of explainable AI and model interpretability has gained added impetus. Methods to extract and communicate visibility into decision-making models have become legal requirement. Two specific types of explanations, contrastive and counterfactual have been identified as suitable for human understanding. In this paper, we propose a model agnostic method and its systemic implementation to generate these explanations using shapely additive explanations (SHAP). We discuss a generative pipeline to create contrastive explanations and use it to further to generate coun…
CardioForest: An Explainable Ensemble Learning Model for Automatic Wide QRS Complex Tachycardia Diagnosis from ECG
School of Artificial Intelligence and Computer Science（人工智能与计算机科学学院）；Nantong University（南通大学）；College of Information Science and Engineering（信息科学与工程学院）；Henan University of Technology（河南理工大学）；Affiliated Hospital of Nantong University（南通大学附属医院）
University of Massachusetts Amherst（马萨诸塞大学阿默斯特分校）；University of California, San Diego（加州大学圣迭戈分校）
There has been a recent resurgence of interest in explainable artificial intelligence (XAI) that aims to reduce the opaqueness of AI-based decision-making systems, allowing humans to scrutinize and trust them. Prior work in this context has focused on the attribution of responsibility for an algorithm's decisions to its inputs wherein responsibility is typically approached as a purely associational concept. In this paper, we propose a principled causality-based approach for explaining black-box decision-making systems that addresses limitations of existing methods in XAI. At the core of our fr…
Counterfactual instances offer human-interpretable insight into the local behaviour of machine learning models. We propose a general framework to generate sparse, in-distribution counterfactual model explanations which match a desired target prediction with a conditional generative model, allowing batches of counterfactual instances to be generated with a single forward pass. The method is flexible with respect to the type of generative model used as well as the task of the underlying predictive model. This allows straightforward application of the framework to different modalities such as ima…
Feature attributions are a common paradigm for model explanations due to their simplicity in assigning a single numeric score for each input feature to a model. In the actionable recourse setting, wherein the goal of the explanations is to improve outcomes for model consumers, it is often unclear how feature attributions should be correctly used. With this work, we aim to strengthen and clarify the link between actionable recourse and feature attributions. Concretely, we propose a variant of SHAP, Counterfactual SHAP (CF-SHAP), that incorporates counterfactual information to produce a backgrou…
Explainable Parallel CNN-LSTM Model for Differentiating Ventricular Tachycardia from Supraventricular Tachycardia with Aberrancy in 12-Lead ECGs
Cardiac Rehabilitation Research Center（心脏康复研究中心）；Cardiovascular Research Institute（心血管研究所）；Department of Cardiology（心内科部）；School of Medicine（医学院）；University of Isfahan（伊斯法罕大学）；Department of Computer Science（计算机科学系）；Faculty of Engineering（工程学院）；Department of Electrical, Computer and Biomedical Engineering（电气、计算机与生物医学工程系）；Toronto Metropolitan University（多伦多 Metropolitan 大学）；Faculty of Electrical Engineering（电气工程学院）；K.N. Toosi University of Technology（K.N. Toosi 技术大学）
The analysis of electrocardiogram (ECG) signals can be time consuming as it is performed manually by cardiologists. Therefore, automation through machine learning (ML) classification is being increasingly proposed which would allow ML models to learn the features of a heartbeat and detect abnormalities. The lack of interpretability hinders the application of Deep Learning in healthcare. Through interpretability of these models, we would understand how a machine learning algorithm makes its decisions and what patterns are being followed for classification. This thesis builds Convolutional Neura…
University Of California, San Diego（加州大学圣地亚哥分校）；Stanford University（斯坦福大学）
With the widespread use of sophisticated machine learning models in sensitive applications, understanding their decision-making has become an essential task. Models trained on tabular data have witnessed significant progress in explanations of their underlying decision making processes by virtue of having a small number of discrete features. However, applying these methods to high-dimensional inputs such as images is not a trivial task. Images are composed of pixels at an atomic level and do not carry any interpretability by themselves. In this work, we seek to use annotated high-level interpr…
Utah State University（犹他州立大学）；New Mexico State University（新墨西哥州立大学）
As machine learning and deep learning models have become highly prevalent in a multitude of domains, the main reservation in their adoption for decision-making processes is their black-box nature. The Explainable Artificial Intelligence (XAI) paradigm has gained a lot of momentum lately due to its ability to reduce models opacity. XAI methods have not only increased stakeholders' trust in the decision process but also helped developers ensure its fairness. Recent efforts have been invested in creating transparent models and post-hoc explanations. However, fewer methods have been developed for …
Objective: Global (inter-patient) ECG classification for arrhythmia detection over Electrocardiogram (ECG) signal is a challenging task for both humans and machines. The main reason is the significant variations of both normal and arrhythmic ECG patterns among patients. Automating this process with utmost accuracy is, therefore, highly desirable due to the advent of wearable ECG sensors. However, even with numerous deep learning approaches proposed recently, there is still a notable gap in the performance of global and patient-specific ECG classification performances. This study proposes a nov…
Counterfactual explanations focus on "actionable knowledge" to help end-users understand how a machine learning outcome could be changed to a more desirable outcome. For this purpose a counterfactual explainer needs to discover input dependencies that relate to outcome changes. Identifying the minimum subset of feature changes needed to action an output change in the decision is an interesting challenge for counterfactual explainers. The DisCERN algorithm introduced in this paper is a case-based counter-factual explainer. Here counterfactuals are formed by replacing feature values from a neare…
We examine counterfactual explanations for explaining the decisions made by model-based AI systems. The counterfactual approach we consider defines an explanation as a set of the system's data inputs that causally drives the decision (i.e., changing the inputs in the set changes the decision) and is irreducible (i.e., changing any subset of the inputs does not change the decision). We (1) demonstrate how this framework may be used to provide explanations for decisions made by general, data-driven AI systems that may incorporate features with arbitrary data types and multiple predictive models,…
There exists an apparent negative correlation between performance and interpretability of deep learning models. In an effort to reduce this negative correlation, we propose a Born Identity Network (BIN), which is a post-hoc approach for producing multi-way counterfactual maps. A counterfactual map transforms an input sample to be conditioned and classified as a target label, which is similar to how humans process knowledge through counterfactual thinking. For example, a counterfactual map can localize hypothetical abnormalities from a normal brain image that may cause it to be diagnosed with a…
A Multimodal and Explainable Machine Learning Approach to Diagnosing Multi-Class Ejection Fraction from Electrocardiograms
Operations Research Center, Massachusetts Institute of Technology（麻省理工学院运营研究中心）；Wisconsin School of Business, University of Wisconsin–Madison（威斯康星大学麦迪逊分校商学院）；Heart and Vascular Institute, Hartford Hospital（哈特福德医院心脏与血管研究院）
ArrhythmiaVision: Resource-Conscious Deep Learning Models with Visual Explanations for ECG Arrhythmia Classification
Dipartimento di Informatica, Università di Verona, Verona, Italy（信息学院，威尼斯大学，威尼斯，意大利）；Department of Computer Science, School of Mathematics and Computer Science, Institute of Business Administration, Karachi, Pakistan（计算机科学系，数学与计算机科学学院，商学院，卡拉奇，巴基斯坦）；Faculty of Information Technology, Salim Habib University, Karachi, Pakistan（信息技术学院，萨利姆·哈比大学，卡拉奇，巴基斯坦）
Comprehensive Signal Quality Evaluation of a Wearable Textile ECG Garment: A Sex-Balanced Study
Department Digital Health and Analytics, Fraunhofer IIS, Fraunhofer Institute for Integrated Circuits IIS（数字健康与分析部门，弗劳恩霍夫研究所IIS，弗劳恩霍夫集成电路研究所）；Department Artificial Intelligence in Biomedical Engineering, Friedrich-Alexander-University Erlangen Nuremberg（生物医学工程中的人工智能部门，埃尔兰根-纽伦堡弗赖堡-亚历山大大学）
Max Planck Institute for Intelligent Systems（马克斯·普朗克智能系统研究所）；University of Tübingen（蒂宾根大学）
Neural networks are prone to learning shortcuts -- they often model simple correlations, ignoring more complex ones that potentially generalize better. Prior works on image classification show that instead of learning a connection to object shape, deep classifiers tend to exploit spurious correlations with low-level texture or the background for solving the classification task. In this work, we take a step towards more robust and interpretable classifiers that explicitly expose the task's causal structure. Building on current advances in deep generative modeling, we propose to decompose the im…
The complexity of state-of-the-art modeling techniques for image classification impedes the ability to explain model predictions in an interpretable way. Existing explanation methods generally create importance rankings in terms of pixels or pixel groups. However, the resulting explanations lack an optimal size, do not consider feature dependence and are only related to one class. Counterfactual explanation methods are considered promising to explain complex model decisions, since they are associated with a high degree of human interpretability. In this paper, SEDC is introduced as a model-agn…
Microsoft Research India（微软印度研究院）；University of Chicago（芝加哥大学）
Feature attributions and counterfactual explanations are popular approaches to explain a ML model. The former assigns an importance score to each input feature, while the latter provides input examples with minimal changes to alter the model's predictions. To unify these approaches, we provide an interpretation based on the actual causality framework and present two key results in terms of their use. First, we present a method to generate feature attribution explanations from a set of counterfactual examples. These feature attributions convey how important a feature is to changing the classifi…
The semantically disentangled latent subspace in GAN provides rich interpretable controls in image generation. This paper includes two contributions on semantic latent subspace analysis in the scenario of face generation using StyleGAN2. First, we propose a novel approach to disentangle latent subspace semantics by exploiting existing face analysis models, e.g., face parsers and face landmark detectors. These models provide the flexibility to construct various criterions with very concrete and interpretable semantic meanings (e.g., change face shape or change skin color) to restrict latent sub…
Class-Aware Reinforcement Learning for Counterfactual Explanation Generation
Institute of Business Administration Karachi（卡拉奇工商管理学院）；Commonwealth Bank of Australia（澳大利亚联邦银行）；Stanford University（斯坦福大学）；University of New South Wales（新南威尔士大学）
Counterfactual Explanations for Deep Two-Sample Testing
Hasso-Plattner-Institute, University of Potsdam（波茨坦大学洪堡-劳恩堡研究所）；Hasso Plattner Institute for Digital Health at Mount Sinai Icahn School of Medicine at Mount Sinai（辛辛那提医学院洪堡数字健康研究所）
University of California, Irvine（加州大学欧文分校）；Harvard University（哈佛大学）
As machine learning models are increasingly used in critical decision-making settings (e.g., healthcare, finance), there has been a growing emphasis on developing methods to explain model predictions. Such \textit{explanations} are used to understand and establish trust in models and are vital components in machine learning pipelines. Though explanations are a critical piece in these systems, there is little understanding about how they are vulnerable to manipulation by adversaries. In this paper, we discuss how two broad classes of explanations are vulnerable to manipulation. We demonstrate h…
University of Alberta（阿尔伯塔大学）；Canadian VIGOUR Centre（加拿大活力中心）
The 12-lead electrocardiogram (ECG) is a commonly used tool for detecting cardiac abnormalities such as atrial fibrillation, blocks, and irregular complexes. For the PhysioNet/CinC 2020 Challenge, we built an algorithm using gradient boosted tree ensembles fitted on morphology and signal processing features to classify ECG diagnosis. For each lead, we derive features from heart rate variability, PQRST template shape, and the full signal waveform. We join the features of all 12 leads to fit an ensemble of gradient boosting decision trees to predict probabilities of ECG instances belonging to ea…
Exploring the Relationships Between Physiological Signals During Automated Fatigue Detection
Department One（部门一）；Department Two（部门二）；Institute for Intelligent Systems Research（智能系统研究与创新研究所）；School of Mathematics, Physics and Computing（数学、物理与计算学院）；University of Southern Queensland（南方昆士兰大学）
