Interpreting the inner function of neural networks is crucial for the trustworthy development and deployment of these black-box models. Prior interpretability methods focus on correlation-based measures to attribute model decisions to individual examples. However, these measures are susceptible to noise and spurious correlations encoded in the model during the training phase (e.g., biased inputs, model overfitting, or misspecification). Moreover, this process has proven to result in noisy and unstable attributions that prevent any transparent understanding of the model's behavior. In this paper, we develop a robust interventional-based method grounded by causal analysis to capture cause-effect mechanisms in pre-trained neural networks and their relation to the prediction. Our novel approach relies on path interventions to infer the causal mechanisms within hidden layers and isolate relevant and necessary information (to model prediction), avoiding noisy ones. The result is task-specific causal explanatory graphs that can audit model behavior and express the actual causes underlying its performance. We apply our method to vision models trained on classification tasks. On image classification tasks, we provide extensive quantitative experiments to show that our approach can capture more stable and faithful explanations than standard attribution-based methods. Furthermore, the underlying causal graphs reveal the neural interactions in the model, making it a valuable tool in other applications (e.g., model repair).
Causality is Key for Interpretability Claims to Generalise
Mila - Qu\'ebec AI Institute \& Universit\'e de Montr\'eal；Boston University；Cold Spring Harbor Laboratory；Max-Planck-Institute for Intelligent Systems, ELLIS Institute T\"ubingen, University of T\"ubingen
A Multi-Level Causal Intervention Framework for Mechanistic Interpretability in Variational Autoencoders
Indian Institute of Technology, Patna（印度理工学院巴特那分校）；Rajarshi School of Management Technology, Varanasi（拉贾尔希管理技术学院瓦拉纳西分校）；Jaypee Institute of Information Technology, Noida（杰皮信息技术学院诺伊达分校）
Interpretability and causal discovery of the machine learning models to predict the production of CBM wells after hydraulic fracturing
Southwest Petroleum University（西南石油大学）；Institute for Artificial Intelligence, Southwest Petroleum University（西南石油大学人工智能研究院）；State Key Laboratory of Oil and Gas Reservoir Geology and Exploitation（油气藏地质及开发工程国家重点实验室）
The Quest for the Right Mediator: Surveying Mechanistic Interpretability Through the Lens of Causal Mediation Analysis
Boston University（波士顿大学）；University of Mannheim（曼海姆大学）；Northeastern University（东北大学）；Anthropic（Anthropic公司）；Massachusetts Institute of Technology（麻省理工学院）；Stanford University（斯坦福大学）；Technion – Israel Institute of Technology（技术学院–以色列理工学院）
Causal Inference via Nonlinear Variable Decorrelation for Healthcare Applications
University of Massachusetts Amherst（马萨诸塞大学阿默斯特分校）；University of Rochester（罗切斯特大学）；Shanghai Maritime University（上海海事大学）；University of Rochester Medical Center（罗切斯特大学医学中心）
Posterior-Calibrated Causal Circuits in Variational Autoencoders: Why Image-Domain Interpretability Fails on Tabular Data
Indian Institute of Technology, Patna（印度理工学院巴特那分校）；Rajarshi School of Management Technology, Varanasi（拉贾尔希管理技术学院瓦拉纳西分校）；Jaypee Institute of Information Technology, Noida（杰皮信息技术学院诺伊达分校）
Mechanistic Interpretability of Brain-to-Speech Models Across Speech Modes
Department of Computer Science, University of Maryland, College Park, United States（计算机科学系，马里兰大学，College Park，美国）；Department of Electrical and Computer Engineering, University of Maryland, College Park, United States（电气与计算机工程系，马里兰大学，College Park，美国）
Causality and Interpretability for Electrical Distribution System faults
ABB Ability Innovation Center（ABB Ability创新中心）；Department of Electronics and Communication Engineering（电子与通信工程系）；IIT Bhubaneswar（比哈尔邦理工学院）；Department of Artificial Intelligence（人工智能系）；IIT Hyderabad（海得拉巴理工学院）；Department of Computer Science and Engineering（计算机科学与工程系）
Boston University（波士顿大学）；Pr(AI)2 R Group（Pr(AI)2 R 研究组）；Ai2（艾伦人工智能研究所）；Technion（以色列理工学院）；University of Buenos Aires（布宜诺斯艾利斯大学）；Northeastern University（东北大学）；Brown University（布朗大学）；University of Amsterdam（阿姆斯特丹大学）；MIT（麻省理工学院）；Cambridge University（剑桥大学）
本文提出机制可解释性基准 MIB，通过电路定位与因果变量定位两赛道系统比较方法，发现归因/掩码优化及有监督 DAS 分别表现最佳。
Emerging Synergies in Causality and Deep Generative Models: A Survey
University of New South Wales（新南威尔士大学）；Carnegie Mellon University（卡内基梅隆大学）；Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）；University of California, San Diego（加利福尼亚大学圣迭戈分校）；CSIRO’s Data61（澳大利亚联邦科学与工业研究组织Data61研究院）
CRTRE: Causal Rule Generation with Target Trial Emulation Framework
University of Massachusetts Amherst（马萨诸塞大学阿默斯特分校）；University of Rochester（罗切斯特大学）；Shanghai Maritime University（上海海事大学）；University of Massachusetts Lowell（马萨诸塞大学洛厄尔分校）
