Deep neural networks are complex and opaque. As they enter application in a variety of important and safety critical domains, users seek methods to explain their output predictions. We develop an approach to explaining deep neural networks by constructing causal models on salient concepts contained in a CNN. We develop methods to extract salient concepts throughout a target network by using autoencoders trained to extract human-understandable representations of network activations. We then build a bayesian causal model using these extracted concepts as variables in order to explain image classification. Finally, we use this causal model to identify and visualize features with significant causal influence on final classification.
Indiana University（印第安纳大学）；Electronics and Telecommunications Research Institute（韩国电子通信研究院）
An autoencoder-based codec employs quantization to turn its bottleneck layer activation into bitstrings, a process that hinders information flow between the encoder and decoder parts. To circumvent this issue, we employ additional skip connections between the corresponding pair of encoder-decoder layers. The assumption is that, in a mirrored autoencoder topology, a decoder layer reconstructs the intermediate feature representation of its corresponding encoder layer. Hence, any additional information directly propagated from the corresponding encoder layer helps the reconstruction. We implement…
Causal modeling has been recognized as a potential solution to many challenging problems in machine learning (ML). Here, we describe how a recently proposed counterfactual approach developed to deconfound linear structural causal models can still be used to deconfound the feature representations learned by deep neural network (DNN) models. The key insight is that by training an accurate DNN using softmax activation at the classification layer, and then adopting the representation learned by the last layer prior to the output layer as our features, we have that, by construction, the learned fea…
Georgia Institute of Technology（佐治亚理工学院）；IBM Research（IBM研究院）
Discovering and exploiting the causality in deep neural networks (DNNs) are crucial challenges for understanding and reasoning causal effects (CE) on an explainable visual model. "Intervention" has been widely used for recognizing a causal relation ontologically. In this paper, we propose a causal inference framework for visual reasoning via do-calculus. To study the intervention effects on pixel-level features for causal reasoning, we introduce pixel-wise masking and adversarial perturbation. In our framework, CE is calculated using features in a latent space and perturbed prediction from a D…
Tübingen AI Center（图宾根人工智能中心）；University of Tübingen（图宾根大学）；Technical University of Darmstadt（达姆施塔特工业大学）
A precise understanding of why units in an artificial network respond to certain stimuli would constitute a big step towards explainable artificial intelligence. One widely used approach towards this goal is to visualize unit responses via activation maximization. These synthetic feature visualizations are purported to provide humans with precise information about the image features that cause a unit to be activated - an advantage over other alternatives like strongly activating natural dataset samples. If humans indeed gain causal insight from visualizations, this should enable them to predic…
Mechanistic Interpretability of Brain-to-Speech Models Across Speech Modes
Department of Computer Science, University of Maryland, College Park, United States（计算机科学系，马里兰大学，College Park，美国）；Department of Electrical and Computer Engineering, University of Maryland, College Park, United States（电气与计算机工程系，马里兰大学，College Park，美国）
Diagnosing and Repairing Unsafe Channels in Vision-Language Models via Causal Discovery and Dual-Modal Safety Subspace Projection
Beijing University of Posts and Telecommunications（北京邮电大学）；Huawei Technologies Ltd.（华为技术有限公司）；Peking University（北京大学）；Chongqing University of Posts and Telecommunications（重庆邮电大学）
From Human Cognition to Neural Activations: Probing the Computational Primitives of Spatial Reasoning in LLMs
School of Information Science, Beijing Language and Culture University（北京语言大学信息科学学院）；School of Artificial Intelligence, Beijing Normal University（北京师范大学人工智能学院）
