Centrale Marseille（马赛中央理工学院）；Aix Marseille Univ（艾克斯-马赛大学）；CNRS（法国国家科学研究中心）；LIS（信息科学与系统实验室）；Institute of Advanced Research on Artificial Intelligence (IARAI)（人工智能高级研究院（IARAI））
This paper studies interpretability of convolutional networks by means of saliency maps. Most approaches based on Class Activation Maps (CAM) combine information from fully connected layers and gradient through variants of backpropagation. However, it is well understood that gradients are noisy and alternatives like guided backpropagation have been proposed to obtain better visualization at inference. In this work, we present a novel training approach to improve the quality of gradients for interpretability. In particular, we introduce a regularization loss such that the gradient with respect to the input image obtained by standard backpropagation is similar to the gradient obtained by guided backpropagation. We find that the resulting gradient is qualitatively less noisy and improves quantitatively the interpretability properties of different networks, using several interpretability methods.
Adversarial robust models have been shown to learn more robust and interpretable features than standard trained models. As shown in [\cite{tsipras2018robustness}], such robust models inherit useful interpretable properties where the gradient aligns perceptually well with images, and adding a large targeted adversarial perturbation leads to an image resembling the target class. We perform experiments to show that interpretable and perceptually aligned gradients are present even in models that do not show high robustness to adversarial attacks. Specifically, we perform adversarial training with …
Max Planck Institute of Molecular Cell Biology and Genetics（马克斯·普朗克分子细胞生物学与遗传学研究所）；Center for Systems Biology Dresden（德累斯顿系统生物学中心）
Convolutional neural networks have become state-of-the-art in a wide range of image recognition tasks. The interpretation of their predictions, however, is an active area of research. Whereas various interpretation methods have been suggested for image classification, the interpretation of image segmentation still remains largely unexplored. To that end, we propose SEG-GRAD-CAM, a gradient-based method for interpreting semantic segmentation. Our method is an extension of the widely-used Grad-CAM method, applied locally to produce heatmaps showing the relevance of individual pixels for semantic…
Pittsburgh, PA 15213, USA（美国宾夕法尼亚州匹兹堡（注：该地址不属于机构，可能是原数据错误，按要求应忽略，此处仅为示例说明））
Recent work has found that adversarially-robust deep networks used for image classification are more interpretable: their feature attributions tend to be sharper, and are more concentrated on the objects associated with the image's ground-truth class. We show that smooth decision boundaries play an important role in this enhanced interpretability, as the model's input gradients around data points will more closely align with boundaries' normal vectors when they are smooth. Thus, because robust models have smoother boundaries, the results of gradient-based attribution methods, like Integrated G…
Hubei University of Technology（湖北工业大学）；Central China Normal University（华中师范大学）
The black-box nature of Deep Neural Networks (DNNs) severely hinders its performance improvement and application in specific scenes. In recent years, class activation mapping-based method has been widely used to interpret the internal decisions of models in computer vision tasks. However, when this method uses backpropagation to obtain gradients, it will cause noise in the saliency map, and even locate features that are irrelevant to decisions. In this paper, we propose an Absolute value Class Activation Mapping-based (Abs-CAM) method, which optimizes the gradients derived from the backpropaga…
As Convolutional Neural Networks embed themselves into our everyday lives, the need for them to be interpretable increases. However, there is often a trade-off between methods that are efficient to compute but produce an explanation that is difficult to interpret, and those that are slow to compute but provide a more interpretable result. This is particularly challenging in problem spaces that require a large input volume, especially video which combines both spatial and temporal dimensions. In this work we introduce the idea of scoring superpixels through the use of gradient based pixel scori…
Bosch Center for Artificial Intelligence (BCAI)（博世人工智能中心）；University of Bonn（波恩大学）
To verify and validate networks, it is essential to gain insight into their decisions, limitations as well as possible shortcomings of training data. In this work, we propose a post-hoc, optimization based visual explanation method, which highlights the evidence in the input image for a specific prediction. Our approach is based on a novel technique to defend against adversarial evidence (i.e. faulty evidence due to artefacts) by filtering gradients during optimization. The defense does not depend on human-tuned parameters. It enables explanations which are both fine-grained and preserve the c…
Paulson School of Engineering and Applied Sciences, Harvard University（哈佛大学保尔森工程与应用科学学院）
Deep neural networks have proven remarkably effective at solving many classification problems, but have been criticized recently for two major weaknesses: the reasons behind their predictions are uninterpretable, and the predictions themselves can often be fooled by small adversarial perturbations. These problems pose major obstacles for the adoption of neural networks in domains that require security or transparency. In this work, we evaluate the effectiveness of defenses that differentiably penalize the degree to which small changes in inputs can alter model predictions. Across multiple atta…
Although neural networks have seen tremendous success as predictive models in a variety of domains, they can be overly confident in their predictions on out-of-distribution (OOD) data. To be viable for safety-critical applications, like autonomous vehicles, neural networks must accurately estimate their epistemic or model uncertainty, achieving a level of system self-awareness. Techniques for epistemic uncertainty quantification often require OOD data during training or multiple neural network forward passes during inference. These approaches may not be suitable for real-time performance on hi…
Bosch Center for Artificial Intelligence（博世人工智能中心）；Max Planck Institute for Intelligent Systems（马克斯·普朗克智能系统研究所）；Max Planck ETH Center for Learning Systems（马克斯·普朗克ETH学习系统中心）；University of Mannheim（曼海姆大学）
Max Planck Institute of Animal Behavior（马克斯·普朗克动物行为研究所）；University of Konstanz（康斯坦茨大学）；Centro de Investigaciones Científicas de las Huastecas Aguazarca, A.C.（华斯特卡斯阿瓜扎尔卡科学研究中心）；University of Würzburg（维尔茨堡大学）；Texas A&M University（得克萨斯农工大学）；Texas State University San Marcos（得克萨斯州立大学圣马科斯分校）
Apart from discriminative models for classification and object detection tasks, the application of deep convolutional neural networks to basic research utilizing natural imaging data has been somewhat limited; particularly in cases where a set of interpretable features for downstream analysis is needed, a key requirement for many scientific investigations. We present an algorithm and training paradigm designed specifically to address this: decontextualized hierarchical representation learning (DHRL). By combining a generative model chaining procedure with a ladder network architecture and late…
Technical University of Munich（慕尼黑工业大学）；Johns Hopkins University（约翰斯·霍普金斯大学）
Neural networks are proven to be remarkably successful for classification and diagnosis in medical applications. However, the ambiguity in the decision-making process and the interpretability of the learned features is a matter of concern. In this work, we propose a method for improving the feature interpretability of neural network classifiers. Initially, we propose a baseline convolutional neural network with state of the art performance in terms of accuracy and weakly supervised localization. Subsequently, the loss is modified to integrate robustness to adversarial examples into the trainin…
Lehigh University（里海大学）；University of Pennsylvania（宾夕法尼亚大学）
Identification of brain regions related to the specific neurological disorders are of great importance for biomarker and diagnostic studies. In this paper, we propose an interpretable Graph Convolutional Network (GCN) framework for the identification and classification of Alzheimer's disease (AD) using multi-modality brain imaging data. Specifically, we extended the Gradient Class Activation Mapping (Grad-CAM) technique to quantify the most discriminative features identified by GCN from brain connectivity patterns. We then utilized them to find signature regions of interest (ROIs) by detecting…
To New Beginnings: A Survey of Unified Perception in Autonomous Vehicle Software
Technical University of Munich（慕尼黑技术大学）；School of Engineering & Design（工程与设计学院）；Department of Mobility Systems Engineering（移动系统工程系）；Institute of Automotive Technology（汽车技术研究所）；Munich Institute of Robotics and Machine Intelligence (MIRMI)（慕尼黑机器人与智能机械研究所）
DRIVE: Dependable Robust Interpretable Visionary Ensemble Framework in Autonomous Driving
Tongji University（同济大学）；King Abdullah University of Science and Technology（阿卜杜拉国王科技大学）；The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；University of Macau（澳门大学）
Peter Munk Cardiac Center（彼得·芒克心脏中心）；University of Toronto（多伦多大学）；Vector Institute（向量研究所）；University of Waterloo（滑铁卢大学）
Medical image segmentation is a difficult but important task for many clinical operations such as cardiac bi-ventricular volume estimation. More recently, there has been a shift to utilizing deep learning and fully convolutional neural networks (CNNs) to perform image segmentation that has yielded state-of-the-art results in many public benchmark datasets. Despite the progress of deep learning in medical image segmentation, standard CNNs are still not fully adopted in clinical settings as they lack robustness and interpretability. Shapes are generally more meaningful features than solely textu…
SurfAge-Net: A Hierarchical Surface-Based Network for Interpretable Fine-Grained Brain Age Prediction
Gansu Provincial Key Laboratory of Wearable Computing, School of Information Science and Engineering, Lanzhou University（甘肃省可穿戴计算重点实验室，信息科学与工程学院，兰州大学）；Department of Medical Imaging Center, Gansu Provincial Maternity and Child-Care Hospital（医学影像中心，甘肃省妇幼保健院）；Department of Magnetic Resonance, Lanzhou University Second Hospital（磁共振科，兰州大学第二医院）；Department of Computer Science and Technology, Harbin Institute of Technology (Shenzhen)（计算机科学与技术系，哈尔滨工业大学（深圳））；Key Laboratory for Biomedical Engineering of Ministry of Education, Department of Biomedical Engineering, College of Biomedical Engineering & Instrument Science, Zhejiang University（教育部生物医学工程重点实验室，生物医学工程学院，浙江大学）
