Looking in the mirror: A faithful counterfactual explanation method for interpreting deep image classification models
Australian Institute for Machine Learning, University of Adelaide, Australia（澳大利亚机器学习研究所，阿德莱德大学，澳大利亚）；Deakin University, Australia（德肯大学，澳大利亚）；Flinders University, Australia（弗林德斯大学，澳大利亚）
Townim Faisal Chowdhury, Vu Minh Hieu Phan, Kewen Liao, Nanyu Dong, Minh-Son To, Anton Hengel, Johan Verjans, Zhibin Liao
备注：
Accepted at IEEE/CVF International Conference on Computer Vision (ICCV), 2025
Counterfactual explanations (CFE) for deep image classifiers aim to reveal how minimal input changes lead to different model decisions, providing critical insights for model interpretation and improvement. However, existing CFE methods often rely on additional image encoders and generative models to create plausible images, neglecting the classifier's own feature space and decision boundaries. As such, they do not explain the intrinsic feature space and decision boundaries learned by the classifier. To address this limitation, we propose Mirror-CFE, a novel method that generates faithful counterfactual explanations by operating directly in the classifier's feature space, treating decision boundaries as mirrors that ``reflect'' feature representations in the mirror. Mirror-CFE learns a mapping function from feature space to image space while preserving distance relationships, enabling smooth transitions between source images and their counterfactuals. Through extensive experiments on four image datasets, we demonstrate that Mirror-CFE achieves superior performance in validity while maintaining input resemblance compared to state-of-the-art explanation methods. Finally, mirror-CFE provides interpretable visualization of the classifier's decision process by generating step-wise transitions that reveal how features evolve as classification confidence changes.
Nanyang Technological University（南洋理工大学）；Singapore Management University（新加坡管理大学）；Damo Academy Alibaba Group（阿里巴巴达摩院）
We present a novel counterfactual framework for both Zero-Shot Learning (ZSL) and Open-Set Recognition (OSR), whose common challenge is generalizing to the unseen-classes by only training on the seen-classes. Our idea stems from the observation that the generated samples for unseen-classes are often out of the true distribution, which causes severe recognition rate imbalance between the seen-class (high) and unseen-class (low). We show that the key reason is that the generation is not Counterfactual Faithful, and thus we propose a faithful one, whose generation is from the sample-specific coun…
Faithful Action-unit Causal Reasoning for Counterfactually Faithful Emotion Explanations
Faculty of CSE, Ho Chi Minh City University of Technology (HCMUT), VNUHCM（胡志明市理工大学计算机科学与工程学院，越南国家大学胡志明市分校）；Dept. of AI, FPT University（FPT大学人工智能系）；Faculty of DSAI, College of Technology, National Economic University（国民经济大学技术学院数据科学与人工智能系）；Dept. of AI Convergence, Chonnam National University（全南大学人工智能融合系）
Causal-Adapter: Taming Text-to-Image Diffusion for Faithful Counterfactual Generation
Centre for AI, DS\&AI, Astrazeneca, UK（英国阿斯利康人工智能中心）；Institute for Imaging, Data and Communications (IDCOM), School of Engineering, University of Edinburgh, Edinburgh, UK（爱丁堡大学工程学院影像、数据与通信研究所）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）
Did Models Learn Sufficiently? Attribution-Guided Training via Subset-Selected Counterfactual Augmentation
School of Cyber Science and Technology, Shenzhen Campus of Sun Yat-sen University（中山大学深圳校区信息科学与技术学院）；PCL；Institute of Information Engineering, CAS（中国科学院信息工程研究所）；University of Chinese Academy of Sciences（中国科学院大学）；Tianjin University（天津大学）；Lanzhou University（兰州大学）；Imperial College London（伦敦帝国理工学院）
City Data Science Institute - City, University of London（伦敦城市大学城市数据科学研究所）
A novel explainable AI method called CLEAR Image is introduced in this paper. CLEAR Image is based on the view that a satisfactory explanation should be contrastive, counterfactual and measurable. CLEAR Image explains an image's classification probability by contrasting the image with a corresponding image generated automatically via adversarial learning. This enables both salient segmentation and perturbations that faithfully determine each segment's importance. CLEAR Image was successfully applied to a medical imaging case study where it outperformed methods such as Grad-CAM and LIME by an a…
Back to the Feature: Explaining Video Classifiers with Video Counterfactual Explanations
Visual Understanding Research Group, Department of Informatics, King’s College London, UK（信息学院视觉理解研究组，伦敦国王学院，英国）；Department of Informatics, King’s College London, UK（信息学院，伦敦国王学院，英国）
Faithful-MR1: Faithful Multimodal Reasoning via Anchoring and Reinforcing Visual Attention
AMAP, Alibaba Group（阿里云实验室，阿里巴巴集团）；University of Chinese Academy of Sciences（中国科学院大学）；Tsinghua University（清华大学）；Nanyang Technological University（南洋理工大学）
Stanford University（斯坦福大学）；Mila, Quebec AI Institute（魁北克人工智能研究所米拉）；Nuance Communications（纽昂斯通讯公司）
Motivation: Traditional image attribution methods struggle to satisfactorily explain predictions of neural networks. Prediction explanation is important, especially in medical imaging, for avoiding the unintended consequences of deploying AI systems when false positive predictions can impact patient care. Thus, there is a pressing need to develop improved models for model explainability and introspection. Specific problem: A new approach is to transform input images to increase or decrease features which cause the prediction. However, current approaches are difficult to implement as they are m…
