With the ongoing rise of machine learning, the need for methods for explaining decisions made by artificial intelligence systems is becoming a more and more important topic. Especially for image classification tasks, many state-of-the-art tools to explain such classifiers rely on visual highlighting of important areas of the input data. Contrary, counterfactual explanation systems try to enable a counterfactual reasoning by modifying the input image in a way such that the classifier would have made a different prediction. By doing so, the users of counterfactual explanation systems are equipped with a completely different kind of explanatory information. However, methods for generating realistic counterfactual explanations for image classifiers are still rare. Especially in medical contexts, where relevant information often consists of textural and structural information, high-quality counterfactual images have the potential to give meaningful insights into decision processes. In this work, we present GANterfactual, an approach to generate such counterfactual image explanations based on adversarial image-to-image translation techniques. Additionally, we conduct a user study to evaluate our approach in an exemplary medical use case. Our results show that, in the chosen medical use-case, counterfactual explanations lead to significantly better results regarding mental models, explanation satisfaction, trust, emotions, and self-efficacy than two state-of-the-art systems that work with saliency maps, namely LIME and LRP.
University of Augsburg（奥格斯堡大学）；Oregon State University（俄勒冈州立大学）
Counterfactual explanations are a common tool to explain artificial intelligence models. For Reinforcement Learning (RL) agents, they answer "Why not?" or "What if?" questions by illustrating what minimal change to a state is needed such that an agent chooses a different action. Generating counterfactual explanations for RL agents with visual input is especially challenging because of their large state spaces and because their decisions are part of an overarching policy, which includes long-term decision-making. However, research focusing on counterfactual explanations, specifically for RL age…
Causal Disentanglement for Robust Long-tail Medical Image Generation
School of Electrical and Information Engineering, Tianjin University, China（天津大学电气与信息工程学院）；Department of Information Engineering and Computer Science, University of Trento, Italy（特伦托大学信息工程与计算机科学系）；Fondazione Bruno Kessler, Italy（布鲁诺·克塞勒基金会）
McGill University（麦吉尔大学）；MILA (Quebec Artificial Intelligence Institute)（魁北克人工智能研究所（MILA））；NeuroRx Research（NeuroRx研究公司）；University of Edinburgh（爱丁堡大学）；The Alan Turing Institute（艾伦·图灵研究所）
The discovery of patient-specific imaging markers that are predictive of future disease outcomes can help us better understand individual-level heterogeneity of disease evolution. In fact, deep learning models that can provide data-driven personalized markers are much more likely to be adopted in medical practice. In this work, we demonstrate that data-driven biomarker discovery can be achieved through a counterfactual synthesis process. We show how a deep conditional generative model can be used to perturb local imaging features in baseline images that are pertinent to subject-specific future…
Generating healthy counterfactuals with denoising diffusion bridge models
Athinoula A. Martinos Center for Biomedical Imaging（阿提诺拉A·马丁诺生物医学成像中心）；Massachusetts General Hospital and Harvard Medical School（麻省总医院和哈佛医学院）；ETH Zurich（苏黎世联邦理工学院）；Computer Science & Artificial Intelligence Lab（计算机科学与人工智能实验室）；Hawkes Institute（哈维斯研究所）；University College London（伦敦大学学院）
MedEdit: Counterfactual Diffusion-based Image Editing on Brain MRI
Technical University of Munich（慕尼黑工业大学）；Helmholtz AI（亥姆霍兹人工智能研究所）；Helmholtz Center Munich（亥姆霍兹慕尼黑中心）；Klinikum Rechts der Isar（慕尼黑 Rechts der Isar 医院）；Kings College London（伦敦国王学院）
Debiasing Counterfactuals In the Presence of Spurious Correlations
MILA (Quebec AI institute)（魁北克人工智能研究院）；Montreal Neurological Institute, McGill University（麦吉尔大学蒙特利尔神经学研究所）；The Alan Turing Institute（艾伦·图灵研究所）；Center for Intelligent Machines, McGill University（麦吉尔大学智能机器中心）；Institute for Digital Communications, School of Engineering, University of Edinburgh（爱丁堡大学工程学院数字通信研究所）
COMSATS University Islamabad（伊斯兰堡COMSATS大学）；National Center of Artificial Intelligence（国家人工智能中心）；University of Strasbourg（斯特拉斯堡大学）；CNRS（法国国家科学研究中心）；Ecole de technologie superieure（高等技术学院）
The image-based diagnosis is now a vital aspect of modern automation assisted diagnosis. To enable models to produce pixel-level diagnosis, pixel-level ground-truth labels are essentially required. However, since it is often not straight forward to obtain the labels in many application domains such as in medical image, classification-based approaches have become the de facto standard to perform the diagnosis. Though they can identify class-salient regions, they may not be useful for diagnosis where capturing all of the evidences is important requirement. Alternatively, a counterfactual explana…
Diffusion Models for Counterfactual Generation and Anomaly Detection in Brain Images
University of Edinburgh（爱丁堡大学）；NHS Lothian（洛锡安国民保健服务体系）；UK Dementia Research Institute Centre, University of Edinburgh（英国痴呆症研究所爱丁堡中心）；University of Dundee（邓迪大学）
Integrating Anatomical Priors into a Causal Diffusion Model
Department of Electrical Engineering, Stanford University, Stanford, CA, USA（电气工程系，斯坦福大学）；Department of Psychiatry and Behavioral Sciences, Stanford University, Stanford, CA, USA（精神病学与行为科学系，斯坦福大学）
Reconstruct or Generate: Exploring the Spectrum of Generative Modeling for Cardiac MRI
School of Computation, Information and Technology（计算信息学院）；Technical University of Munich（慕尼黑技术大学）；Munich Center for Machine Learning（慕尼黑机器学习中心）；School of Medicine, Klinikum Rechts der Isar（医学院，莱茵河右岸医院）；Department of Quantitative Biomedicine（定量生物医学系）；University of Zurich（苏黎世大学）；Department of Computing（计算系）；Imperial College London（伦敦帝国理工学院）
Clinical applicability of automated decision support systems depends on a robust, well-understood classification interpretation. Artificial neural networks while achieving class-leading scores fall short in this regard. Therefore, numerous approaches have been proposed that map a salient region of an image to a diagnostic classification. Utilizing heuristic methodology, like blurring and noise, they tend to produce diffuse, sometimes misleading results, hindering their general adoption. In this work we overcome these issues by presenting a model agnostic saliency mapping framework tailored to …
Babylon Health（巴比伦健康）；University College London（伦敦大学学院）
Machine learning promises to revolutionize clinical decision making and diagnosis. In medical diagnosis a doctor aims to explain a patient's symptoms by determining the diseases \emph{causing} them. However, existing diagnostic algorithms are purely associative, identifying diseases that are strongly correlated with a patients symptoms and medical history. We show that this inability to disentangle correlation from causation can result in sub-optimal or dangerous diagnoses. To overcome this, we reformulate diagnosis as a counterfactual inference task and derive new counterfactual diagnostic al…
AI Alignment in Medical Imaging: Unveiling Hidden Biases Through Counterfactual Analysis
School of Computation, Information and Technology, Technical University of Munich（技术大学慕尼黑计算、信息与技术学院）；Department of Computer Science, University of Oxford（牛津大学计算机科学系）；Department of Engineering Science, University of Oxford（牛津大学工程科学系）；Helmholtz AI, Munich, Germany（海德堡人工智能研究所，德国慕尼黑）
On the notion of missingness for path attribution explainability methods in medical settings: Guiding the selection of medically meaningful baselines
Research Group MITI, Technical University of Munich（慕尼黑技术大学MITI研究组）；Chair for AI in Healthcare and Medicine, Technical University of Munich (TUM) and TUM University Hospital（慕尼黑技术大学人工智能在医疗健康中的chair及慕尼黑技术大学医院）；Department of Computing, Imperial College London（伦敦帝国理工学院计算机系）；Department of Surgery, Technical University of Munich（慕尼黑技术大学外科系）
Counterfactual Explanations for Medical Image Classification and Regression using Diffusion Autoencoder
Technical University of Munich（慕尼黑工业大学）；Klinikum rechts der Isar（右侧伊萨尔医院）；Institute for Artificial Intelligence and Computer Science in Medicine, Technical University of Munich（慕尼黑工业大学人工智能与医学计算机科学研究所）；Computer Aided Medical Procedures, Technical University of Munich（慕尼黑工业大学计算机辅助医疗程序研究所）；AI for Image-Guided Diagnosis and Therapy, Technical University of Munich（慕尼黑工业大学图像引导诊断与治疗人工智能研究所）
VRVis Zentrum für Virtual Reality und Visualisierung Forschungs-GmbH（VRVis虚拟现实与可视化研究有限公司）
The success of machine learning methods for computer vision tasks has driven a surge in computer assisted prediction for medicine and biology. Based on a data-driven relationship between input image and pathological classification, these predictors deliver unprecedented accuracy. Yet, the numerous approaches trying to explain the causality of this learned relationship have fallen short: time constraints, coarse, diffuse and at times misleading results, caused by the employment of heuristic techniques like Gaussian noise and blurring, have hindered their clinical adoption. In this work, we disc…
University of Edinburgh（爱丁堡大学）；The Alan Turing Institute（艾伦·图灵研究所）
Due to the limited availability of medical data, deep learning approaches for medical image analysis tend to generalise poorly to unseen data. Augmenting data during training with random transformations has been shown to help and became a ubiquitous technique for training neural networks. Here, we propose a novel adversarial counterfactual augmentation scheme that aims at finding the most \textit{effective} synthesised images to improve downstream tasks, given a pre-trained generative model. Specifically, we construct an adversarial game where we update the input \textit{conditional factor} of…
University of Oxford（牛津大学）；The Alan Turing Institute（艾伦·图灵研究所）；University of California, Los Angeles（加利福尼亚大学洛杉矶分校）；University of Cambridge（剑桥大学）
Identifying when to give treatments to patients and how to select among multiple treatments over time are important medical problems with a few existing solutions. In this paper, we introduce the Counterfactual Recurrent Network (CRN), a novel sequence-to-sequence model that leverages the increasingly available patient observational data to estimate treatment effects over time and answer such medical questions. To handle the bias from time-varying confounders, covariates affecting the treatment assignment policy in the observational data, CRN uses domain adversarial training to build balancing…
We tackle the problem of computing counterfactual explanations -- minimal changes to the features that flip an undesirable model prediction. We propose a solution to this question for linear Support Vector Machine (SVMs) models. Moreover, we introduce a way to account for weighted actions that allow for more changes in certain features than others. In particular, we show how to find counterfactual explanations with the purpose of increasing model interpretability. These explanations are valid, change only actionable features, are close to the data distribution, sparse, and take into account co…
University College Dublin（都柏林大学学院）；Insight Centre for Data Analytics（洞察数据分析中心）；VistaMilk SFI Research Centre（VistaMilk SFI研究中心）
Whilst an abundance of techniques have recently been proposed to generate counterfactual explanations for the predictions of opaque black-box systems, markedly less attention has been paid to exploring the uncertainty of these generated explanations. This becomes a critical issue in high-stakes scenarios, where uncertain and misleading explanations could have dire consequences (e.g., medical diagnosis and treatment planning). Moreover, it is often difficult to determine if the generated explanations are well grounded in the training data and sensitive to distributional shifts. This paper propo…
Technical University of Munich（慕尼黑工业大学）；Klinikum rechts der Isar（伊萨尔河畔临床教学中心）
Deep learning models used in medical image analysis are prone to raising reliability concerns due to their black-box nature. To shed light on these black-box models, previous works predominantly focus on identifying the contribution of input features to the diagnosis, i.e., feature attribution. In this work, we explore counterfactual explanations to identify what patterns the models rely on for diagnosis. Specifically, we investigate the effect of changing features within chest X-rays on the classifier's output to understand its decision mechanism. We leverage a StyleGAN-based approach (StyleE…
Understanding and explaining the mistakes made by trained models is critical to many machine learning objectives, such as improving robustness, addressing concept drift, and mitigating biases. However, this is often an ad hoc process that involves manually looking at the model's mistakes on many test samples and guessing at the underlying reasons for those incorrect predictions. In this paper, we propose a systematic approach, conceptual counterfactual explanations (CCE), that explains why a classifier makes a mistake on a particular test sample(s) in terms of human-understandable concepts (e.…
Counterfactual explanations (CEs) are a practical tool for demonstrating why machine learning classifiers make particular decisions. For CEs to be useful, it is important that they are easy for users to interpret. Existing methods for generating interpretable CEs rely on auxiliary generative models, which may not be suitable for complex datasets, and incur engineering overhead. We introduce a simple and fast method for generating interpretable CEs in a white-box setting without an auxiliary model, by using the predictive uncertainty of the classifier. Our experiments show that our proposed alg…
Generating Counterfactual Trajectories with Latent Diffusion Models for Concept Discovery
Rheinland-Pfälzische Technische Universität Kaiserslautern-Landau（莱茵兰-普法尔茨州 Kaiserslautern-Landau 技术大学）；German Research Center for Artificial Intelligence GmbH (DFKI)（德国人工智能研究中心 GmbH（DFKI））
Walking the Tightrope: Disentangling Beneficial and Detrimental Drifts in Non-Stationary Custom-Tuning
Australian Artificial Intelligence Institute (AAII)（澳大利亚人工智能研究所）；Faulty of Engineering and Information Technology（工程与信息技术学院）；University of Technology Sydney（悉尼技术大学）
