Concept-based explanations permit to understand the predictions of a deep neural network (DNN) through the lens of concepts specified by users. Existing methods assume that the examples illustrating a concept are mapped in a fixed direction of the DNN's latent space. When this holds true, the concept can be represented by a concept activation vector (CAV) pointing in that direction. In this work, we propose to relax this assumption by allowing concept examples to be scattered across different clusters in the DNN's latent space. Each concept is then represented by a region of the DNN's latent space that includes these clusters and that we call concept activation region (CAR). To formalize this idea, we introduce an extension of the CAV formalism that is based on the kernel trick and support vector classifiers. This CAR formalism yields global concept-based explanations and local concept-based feature importance. We prove that CAR explanations built with radial kernels are invariant under latent space isometries. In this way, CAR assigns the same explanations to latent spaces that have the same geometry. We further demonstrate empirically that CARs offer (1) more accurate descriptions of how concepts are scattered in the DNN's latent space; (2) global explanations that are closer to human concept annotations and (3) concept-based feature importance that meaningfully relate concepts with each other. Finally, we use CARs to show that DNNs can autonomously rediscover known scientific concepts, such as the prostate cancer grading system.
Universal Activation Verbalizer: A Unified Framework for Cross-Model Activation Explanation
New Jersey Institute of Technology（新泽西理工学院）；University of North Carolina at Charlotte（北卡罗来纳大学夏洛特分校）；Cisco Research（思科研究）；The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））
Beyond Activation Patterns: A Weight-Based Out-of-Context Explanation of Sparse Autoencoder Features
State Key Laboratory of General Artificial Intelligence, School of Intelligence Science and Technology, Peking University（通用人工智能国家重点实验室，智能科学与技术学院，北京大学）
A Theoretical Explanation of Activation Sparsity through Flat Minima and Adversarial Robustness
State Key Laboratory for Novel Software Technology（计算机软件新技术全国重点实验室）；Nanjing University（南京大学）；School of Computer Science and Engineering（计算机科学与工程学院）；Southeast University（东南大学）
From Attribution to Action: A Human-Centered Application of Activation Steering
Fraunhofer Heinrich-Hertz-Institut（弗劳恩霍夫 Heinrich-Hertz 研究所）；Technische Universität Berlin（柏林技术大学）；BIFOLD – Berlin Institute for the Foundations of Learning and Data（柏林学习与数据基础研究所）
Counterfactual Explanation of Brain Activity Classifiers using Image-to-Image Transfer by Generative Adversarial Network
Okayama University（冈山大学）；Japan Science and Technology Agency（日本科学技术振兴机构）；Rikkyo University（立教大学）；National Institute for Physiological Sciences（生理学研究所）；Araya Inc.（Araya 公司）；Keio University（庆应义塾大学）
SAEExplainer: Interpreting SAE Features with Activation-Guided Preference Optimization
Shanghai Jiao Tong University（上海交通大学）；NJIT（新泽西理工学院）；Jilin University（吉林大学）；Institute of Computing Technology, CAS（中国科学院计算技术研究所）；The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））
School of Computer Science, Peking University, Beijing 100871, China（北京大学计算机科学学院）；Key Lab of High Confidence Software Technologies (Peking University), Ministry of Education（高可信软件技术重点实验室（北京大学））
UCSF（加州大学旧金山分校）；SFVAMC（旧金山退伍军人事务医疗中心）；UC Berkeley（加州大学伯克利分校）；Helen Diller Family Comprehensive Cancer Center, UCSF（加州大学旧金山分校海伦·迪勒家族综合癌症中心）；Bakar Computational Health Sciences Institute, UCSF（加州大学旧金山分校巴卡尔计算健康科学研究所）
Post-Hoc Concept Disentanglement: From Correlated to Isolated Concept Representations
Fraunhofer Heinrich Hertz Institut（弗劳恩霍夫海因里希·赫兹研究所）；Technological University Dublin（都柏林理工大学）；Technische Universität Berlin（柏林工业大学）；Berlin Institute for the Foundations of Learning and Data (BIFOLD)（柏林学习与数据基础研究所）
Signature Activation: A Sparse Signal View for Holistic Saliency
Harvard University（哈佛大学）；Broad Institute of Massachusetts Institute of Technology and Harvard（麻省理工学院与哈佛大学博德研究所）；Massachusetts General Hospital（麻省总医院）；Harvard Medical School（哈佛医学院）
