Researchers have proposed a wide variety of model explanation approaches, but it remains unclear how most methods are related or when one method is preferable to another. We examine the literature and find that many methods are based on a shared principle of explaining by removing - essentially, measuring the impact of removing sets of features from a model. These methods vary in several respects, so we develop a framework for removal-based explanations that characterizes each method along three dimensions: 1) how the method removes features, 2) what model behavior the method explains, and 3) how the method summarizes each feature's influence. Our framework unifies 26 existing methods, including several of the most widely used approaches (SHAP, LIME, Meaningful Perturbations, permutation tests). Exposing the fundamental similarities between these methods empowers users to reason about which tools to use, and suggests promising directions for ongoing model explainability research.
Unifying local and global model explanations by functional decomposition of low dimensional structures
University of Copenhagen（哥本哈根大学）；Heidelberg University（海德堡大学）；Leibniz Institute for Prevention Research and Epidemiology – BIPS（莱布尼茨预防研究与流行病学研究所（BIPS））；University of Bremen（不来梅大学）
Unifying Image Counterfactuals and Feature Attributions with Latent-Space Adversarial Attacks
Department of Statistics, University of California, Berkeley（加州大学伯克利分校统计学系）；Department of Statistics and Data Science, University of Pennsylvania（宾夕法尼亚大学统计学与数据科学系）
Normalized Relevance Measure as a Unifying Framework to Explain Neural Network Latent Structures
Berlin Institute for the Foundations of Learning（学习与数据基础研究院）；Machine Learning Group, Technical University of Berlin（柏林技术大学机器学习组）；Department of Artificial Intelligence, Korea University（韩国大学人工智能系）；Max Planck Institute for Informatics（信息研究所）；Department of Chemistry, Chemical Physics Theory Group, University of Toronto（多伦多大学化学系、化学物理理论组）；Vector Institute for Artificial Intelligence（人工智能矢量研究所）；Acceleration Consortium, University of Toronto（多伦多大学加速联盟）
LIPEx-Locally Interpretable Probabilistic Explanations-To Look Beyond The True Class
Manchester Centre for Robotics and A.I.（曼彻斯特机器人与人工智能中心）；The University of Manchester（曼彻斯特大学）；University of Liverpool（利物浦大学）；Manchester Centre for A.I. Fundamentals（曼彻斯特人工智能基础中心）
