Technical University of Kaiserslautern（凯泽斯劳滕工业大学）；Max Planck Institute for Informatics（马克斯·普朗克信息学研究所）；Technical University of Berlin（柏林工业大学）；Google Research, Brain Team（谷歌研究院大脑团队）；Korea University（高丽大学）
Philipp Liznerski, Lukas Ruff, Robert A. Vandermeulen, Billy Joe Franks, Marius Kloft, Klaus-Robert Müller
备注：
25 pages, published as a conference paper at ICLR 2021
Deep one-class classification variants for anomaly detection learn a mapping that concentrates nominal samples in feature space causing anomalies to be mapped away. Because this transformation is highly non-linear, finding interpretations poses a significant challenge. In this paper we present an explainable deep one-class classification method, Fully Convolutional Data Description (FCDD), where the mapped samples are themselves also an explanation heatmap. FCDD yields competitive detection performance and provides reasonable explanations on common anomaly detection benchmarks with CIFAR-10 and ImageNet. On MVTec-AD, a recent manufacturing dataset offering ground-truth anomaly maps, FCDD sets a new state of the art in the unsupervised setting. Our method can incorporate ground-truth anomaly maps during training and using even a few of these (~5) improves performance significantly. Finally, using FCDD's explanations we demonstrate the vulnerability of deep one-class classification models to spurious image features such as image watermarks.
ECSEL: Explainable Classification via Signomial Equation Learning
Amsterdam Business School, University of Amsterdam, Amsterdam, the Netherlands（阿姆斯特丹大学阿姆斯特丹商学院）；Institute for Informatics, University of Amsterdam（阿姆斯特丹大学信息学院）；Institute for Logic, Language and Computation, University of Amsterdam（阿姆斯特丹大学逻辑、语言与计算研究所）
Lawrence Livermore National Laboratory（劳伦斯利弗莫尔国家实验室）
The scientific community has been increasingly interested in harnessing the power of deep learning to solve various domain challenges. However, despite the effectiveness in building predictive models, fundamental challenges exist in extracting actionable knowledge from deep neural networks due to their opaque nature. In this work, we propose techniques for exploring the behavior of deep learning models by injecting domain-specific actionable attributes as tunable "knobs" in the analysis pipeline. By incorporating the domain knowledge in a generative modeling framework, we are not only able to …
Deep learning applied to weather forecasting has started gaining popularity because of the progress achieved by data-driven models. The present paper compares two different deep learning architectures to perform weather prediction on daily data gathered from 18 cities across Europe and spanned over a period of 15 years. We propose the Deep Attention Unistream Multistream (DAUM) networks that investigate different types of input representations (i.e. tensorial unistream vs. multistream ) as well as the incorporation of the attention mechanism. In particular, we show that adding a self-attention…
Deep learning bears promise for drug discovery, including advanced image analysis, prediction of molecular structure and function, and automated generation of innovative chemical entities with bespoke properties. Despite the growing number of successful prospective applications, the underlying mathematical models often remain elusive to interpretation by the human mind. There is a demand for 'explainable' deep learning methods to address the need for a new narrative of the machine language of the molecular sciences. This review summarizes the most prominent algorithmic concepts of explainable …
Deep neural networks are susceptible to small-but-specific adversarial perturbations capable of deceiving the network. This vulnerability can lead to potentially harmful consequences in security-critical applications. To address this vulnerability, we propose a novel metric called \emph{Gradient Similarity} that allows us to capture the influence of training data on test inputs. We show that \emph{Gradient Similarity} behaves differently for normal and adversarial inputs, and enables us to detect a variety of adversarial attacks with a near perfect ROC-AUC of 95-100\%. Even white-box adversari…
Universidad Nacional de San Luis（圣路易斯国立大学）；Consejo Nacional de Investigaciones Científicas y Técnicas（国家科学技术研究委员会）；Instituto Nacional de Astrofísica, Óptica y Electrónica（国家天体物理、光学和电子研究所）
A recently introduced text classifier, called SS3, has obtained state-of-the-art performance on the CLEF's eRisk tasks. SS3 was created to deal with risk detection over text streams and, therefore, not only supports incremental training and classification but also can visually explain its rationale. However, little attention has been paid to the potential use of SS3 as a general classifier. We believe this could be due to the unavailability of an open-source implementation of SS3. In this work, we introduce PySS3, a package that implements SS3 and also comes with visualization tools that allow…
Center for Public Health Genomics（公共卫生基因组学中心）；Department of Biomedical Engineering University of Virginia（弗吉尼亚大学生物医学工程系）；University of Virginia（弗吉尼亚大学）
White Blood Cell (WBC) Leukaemia is detected through image-based classification. Convolutional Neural Networks are used to learn the features needed to classify images of cells a malignant or healthy. However, this type of model requires learning a large number of parameters and is difficult to interpret and explain. Explainable AI (XAI) attempts to alleviate this issue by providing insights to how models make decisions. Therefore, we present an XAI model which uses only 24 explainable and interpretable features and is highly competitive to other approaches by outperforming them by about 4.38\…
We propose a method to classify cardiac pathology based on a novel approach to extract image derived features to characterize the shape and motion of the heart. An original semi-supervised learning procedure, which makes efficient use of a large amount of non-segmented images and a small amount of images segmented manually by experts, is developed to generate pixel-wise apparent flow between two time points of a 2D+t cine MRI image sequence. Combining the apparent flow maps and cardiac segmentation masks, we obtain a local apparent flow corresponding to the 2D motion of myocardium and ventricu…
The Open University（开放大学）；Microsoft（微软）；Tel-Aviv University（特拉维夫大学）；Bar-Ilan University（巴伊兰大学）；Technion（以色列理工学院）；Tel Aviv University（特拉维夫大学）
We present Gradient Activation Maps (GAM) - a machinery for explaining predictions made by visual similarity and classification models. By gleaning localized gradient and activation information from multiple network layers, GAM offers improved visual explanations, when compared to existing alternatives. The algorithmic advantages of GAM are explained in detail, and validated empirically, where it is shown that GAM outperforms its alternatives across various tasks and datasets.
University of Southern California（南加州大学）；Columbia University（哥伦比亚大学）；Tencent America（腾讯美国）
An explainable machine learning method for point cloud classification, called the PointHop method, is proposed in this work. The PointHop method consists of two stages: 1) local-to-global attribute building through iterative one-hop information exchange, and 2) classification and ensembles. In the attribute building stage, we address the problem of unordered point cloud data using a space partitioning procedure and developing a robust descriptor that characterizes the relationship between a point and its one-hop neighbor in a PointHop unit. When we put multiple PointHop units in cascade, the a…
Lawrence Livermore National Laboratory（劳伦斯利弗莫尔国家实验室）
In this work, we propose an introspection technique for deep neural networks that relies on a generative model to instigate salient editing of the input image for model interpretation. Such modification provides the fundamental interventional operation that allows us to obtain answers to counterfactual inquiries, i.e., what meaningful change can be made to the input image in order to alter the prediction. We demonstrate how to reveal interesting properties of the given classifiers by utilizing the proposed introspection approach on both the MNIST and the CelebA dataset.
Deep neural networks (DNs) provide superhuman performance in numerous computer vision tasks, yet it remains unclear exactly which of a DN's units contribute to a particular decision. NeuroView is a new family of DN architectures that are interpretable/explainable by design. Each member of the family is derived from a standard DN architecture by vector quantizing the unit output values and feeding them into a global linear classifier. The resulting architecture establishes a direct, causal link between the state of each unit and the classification decision. We validate NeuroView on standard dat…
Deep learning methods exhibit outstanding performance in synthetic aperture radar (SAR) image interpretation tasks. However, these are black box models that limit the comprehension of their predictions. Therefore, to meet this challenge, we have utilized explainable artificial intelligence (XAI) methods for the SAR image classification task. Specifically, we trained state-of-the-art convolutional neural networks for each polarization format on OpenSARUrban dataset and then investigate eight explanation methods to analyze the predictions of the CNN classifiers of SAR images. These XAI methods a…
Massachusetts General Hospital（麻省总医院）；Harvard Medical School（哈佛医学院）；Singapore Eye Research Institute（新加坡眼科研究所）；Duke-NUS Medical School（杜克-新加坡国立大学医学院）；Institute for Molecular and Clinical Ophthalmology（分子与临床眼科研究所）；National University of Singapore（新加坡国立大学）
In this paper, we propose an explainable and interpretable diabetic retinopathy (ExplainDR) classification model based on neural-symbolic learning. To gain explainability, a highlevel symbolic representation should be considered in decision making. Specifically, we introduce a human-readable symbolic representation, which follows a taxonomy style of diabetic retinopathy characteristics related to eye health conditions to achieve explainability. We then include humanreadable features obtained from the symbolic representation in the disease prediction. Experimental results on a diabetic retinopa…
Deep visual models have widespread applications in high-stake domains. Hence, their black-box nature is currently attracting a large interest of the research community. We present the first survey in Explainable AI that focuses on the methods and metrics for interpreting deep visual models. Covering the landmark contributions along the state-of-the-art, we not only provide a taxonomic organization of the existing techniques, but also excavate a range of evaluation metrics and collate them as measures of different properties of model explanations. Along the insightful discussion on the current …
National Technical University of Athens（雅典国家技术大学）；Athena Research Center（雅典娜研究中心）
Although deep neural networks hold the state-of-the-art in several remote sensing tasks, their black-box operation hinders the understanding of their decisions, concealing any bias and other shortcomings in datasets and model performance. To this end, we have applied explainable artificial intelligence (XAI) methods in remote sensing multi-label classification tasks towards producing human-interpretable explanations and improve transparency. In particular, we utilized and trained deep learning models with state-of-the-art performance in the benchmark BigEarthNet and SEN12MS datasets. Ten XAI m…
University of Tübingen（图宾根大学）；Liebherr-Elektronik GmbH（利勃海尔电子有限公司）；Scheidt MessTechnik（沙伊特测量技术公司）
We present a reformulation of the regression and classification, which aims to validate the result of a machine learning algorithm. Our reformulation simplifies the original problem and validates the result of the machine learning algorithm using the training data. Since the validation of machine learning algorithms must always be explainable, we perform our experiments with the kNN algorithm as well as with an algorithm based on conditional probabilities, which is proposed in this work. For the evaluation of our approach, three publicly available data sets were used and three classification a…
Deep learning methods have been very effective for a variety of medical diagnostic tasks and has even beaten human experts on some of those. However, the black-box nature of the algorithms has restricted clinical use. Recent explainability studies aim to show the features that influence the decision of a model the most. The majority of literature reviews of this area have focused on taxonomy, ethics, and the need for explanations. A review of the current applications of explainable deep learning for different medical imaging tasks is presented here. The various approaches, challenges for clini…
Crime and Security Research Institute（犯罪与安全研究所）；Cardiff University（卡迪夫大学）；School of Engineering（工程学院）
The popularity of Deep Learning for real-world applications is ever-growing. With the introduction of high performance hardware, applications are no longer limited to image recognition. With the introduction of more complex problems comes more and more complex solutions, and the increasing need for explainable AI. Deep Neural Networks for Video tasks are amongst the most complex models, with at least twice the parameters of their Image counterparts. However, explanations for these models are often ill-adapted to the video domain. The current work in explainability for video models is still ove…
Northwestern Polytechnical University（西北工业大学）；Remote Sensing Technology Institute (IMF), German Aerospace Center (DLR)（德国航空航天中心（DLR）遥感技术研究所（IMF））；University POLITEHNICA of Bucharest (UPB)（布加勒斯特理工大学（UPB））
Integrating the special electromagnetic characteristics of Synthetic Aperture Radar (SAR) in deep neural networks is essential in order to enhance the explainability and physics awareness of deep learning. In this paper, we first propose a novel physically explainable convolutional neural network for SAR image classification, namely physics guided and injected learning (PGIL). It comprises three parts: (1) explainable models (XM) to provide prior physics knowledge, (2) physics guided network (PGN) to encode the knowledge into physics-aware features, and (3) physics injected network (PIN) to ad…
MedicalPatchNet: A Patch-Based Self-Explainable AI Architecture for Chest X-ray Classification
Department of Diagnostic and Interventional Radiology of the University Hospital RWTH Aachen（鲁尔大学亚琛医院诊断与介入放射科）；Else Kroener Fresenius Center for Digital Health, Technical University Dresden（德累斯顿技术大学埃勒克罗纳弗雷斯内尔数字健康中心）；Department of Medicine III, University Hospital RWTH Aachen（鲁尔大学亚琛医院第三医学部）；Pathology & Data Analytics, Leeds Institute of Medical Research at St James’s, University of Leeds（利兹大学圣詹姆斯医院医学研究所以及病理学与数据科学）；Department of Medicine I, University Hospital Dresden（德累斯顿大学第一医学部）；Medical Oncology, National Center for Tumor Diseases (NCT), University Hospital Heidelberg（海德堡大学国家肿瘤疾病中心（NCT）肿瘤医学部）
Research Institute for Infrastructure Paradigm Shift, Yachiyo Engineering Co., Ltd.（八千代工程公司基础设施范式转变研究所）
Recent years, typhoon damages has become social problem owing to climate change. In 9 September 2019, Typhoon Faxai passed on the Chiba in Japan, whose damages included with electric provision stop because of strong wind recorded on the maximum 45 meter per second. A large amount of tree fell down, and the neighbor electric poles also fell down at the same time. These disaster features have caused that it took 18 days for recovery longer than past ones. Immediate responses are important for faster recovery. As long as we can, aerial survey for global screening of devastated region would be req…
Friedrich-Alexander University Erlangen-Nürnberg (FAU)（埃尔朗根-纽伦堡大学）
For many applications, utilizing DNNs (Deep Neural Networks) requires their implementation on a target architecture in an optimized manner concerning energy consumption, memory requirement, throughput, etc. DNN compression is used to reduce the memory footprint and complexity of a DNN before its deployment on hardware. Recent efforts to understand and explain AI (Artificial Intelligence) methods have led to a new research area, termed as explainable AI. Explainable AI methods allow us to understand better the inner working of DNNs, such as the importance of different neurons and features. The …
Keen Eye（基恩眼科）；Laboratoire Traitement et Communication de l'Information（信息处理与通信实验室）；Telecom ParisTech（巴黎高等电信学院）；Universite Paris-Saclay（巴黎萨克雷大学）
This article adresses the problem of automatic squamous cells classification for cervical cancer screening using Deep Learning methods. We study different architectures on a public dataset called Herlev dataset, which consists in classifying cells, obtained by cervical pap smear, regarding the severity of the abnormalities they represent. Furthermore, we use an attribution method to understand which cytomorphological features are actually learned as discriminative to classify severity of the abnormalities. Through this paper, we show how we trained a performant classifier: 74.5\% accuracy on s…
The complexity of state-of-the-art modeling techniques for image classification impedes the ability to explain model predictions in an interpretable way. Existing explanation methods generally create importance rankings in terms of pixels or pixel groups. However, the resulting explanations lack an optimal size, do not consider feature dependence and are only related to one class. Counterfactual explanation methods are considered promising to explain complex model decisions, since they are associated with a high degree of human interpretability. In this paper, SEDC is introduced as a model-agn…
Melanoma is a prevalent lethal type of cancer that is treatable if diagnosed at early stages of development. Skin lesions are a typical indicator for diagnosing melanoma but they often led to delayed diagnosis due to high similarities of cancerous and benign lesions at early stages of melanoma. Deep learning (DL) can be used as a solution to classify skin lesion pictures with a high accuracy, but clinical adoption of deep learning faces a significant challenge. The reason is that the decision processes of deep learning models are often uninterpretable which makes them black boxes that are chal…
Fraunhofer Institute for Integrated Circuits IIS（弗劳恩霍夫集成电路研究所IIS）；Nuremberg Institute of Technology Georg Simon Ohm（纽伦堡乔治·西蒙·欧姆技术学院）
Would you trust physicians if they cannot explain their decisions to you? Medical diagnostics using machine learning gained enormously in importance within the last decade. However, without further enhancements many state-of-the-art machine learning methods are not suitable for medical application. The most important reasons are insufficient data set quality and the black-box behavior of machine learning algorithms such as Deep Learning models. Consequently, end-users cannot correct the model's decisions and the corresponding explanations. The latter is crucial for the trustworthiness of machi…
This study explores the use of the Dirichlet Variational Autoencoder (DirVAE) for learning disentangled latent representations of chest X-ray (CXR) images. Our working hypothesis is that distributional sparsity, as facilitated by the Dirichlet prior, will encourage disentangled feature learning for the complex task of multi-label classification of CXR images. The DirVAE is trained using CXR images from the CheXpert database, and the predictive capacity of multi-modal latent representations learned by DirVAE models is investigated through implementation of an auxiliary multi-label classificatio…
Institute for Datability Science (IDS)（数据科学研究所）；Graduate School of Medicine Osaka University（大阪大学医学研究生院）
Few-shot learning (FSL) approaches are usually based on an assumption that the pre-trained knowledge can be obtained from base (seen) categories and can be well transferred to novel (unseen) categories. However, there is no guarantee, especially for the latter part. This issue leads to the unknown nature of the inference process in most FSL methods, which hampers its application in some risk-sensitive areas. In this paper, we reveal a new way to perform FSL for image classification, using visual representations from the backbone model and weights generated by a newly-emerged explainable classi…
No Masks Needed: Explainable AI for Deriving Segmentation from Classification
Dept. Electrical and Electronic Engineering, Imperial College London（帝国理工学院伦敦分校电子与电气工程系）；Centre for Vision, Speech and Signal Processing, University of Surrey（萨里大学视觉、语音与信号处理中心）
Huawei Ireland Research Center（华为爱尔兰研究中心）；Boston University（波士顿大学）；Istituto Italiano di Tecnologia（意大利理工学院）；Naver Labs Europe（欧洲Naver实验室）；Microsoft（微软）；Adobe Research（Adobe研究院）；Università di Verona（维罗纳大学）
Conventionally, AI models are thought to trade off explainability for lower accuracy. We develop a training strategy that not only leads to a more explainable AI system for object classification, but as a consequence, suffers no perceptible accuracy degradation. Explanations are defined as regions of visual evidence upon which a deep classification network makes a decision. This is represented in the form of a saliency map conveying how much each pixel contributed to the network's decision. Our training strategy enforces a periodic saliency-based feedback to encourage the model to focus on the…
Recent technological advances in synthetic data have enabled the generation of images with such high quality that human beings cannot tell the difference between real-life photographs and Artificial Intelligence (AI) generated images. Given the critical necessity of data reliability and authentication, this article proposes to enhance our ability to recognise AI-generated images through computer vision. Initially, a synthetic dataset is generated that mirrors the ten classes of the already available CIFAR-10 dataset with latent diffusion which provides a contrasting set of images for compariso…
Understanding model predictions is critical in healthcare, to facilitate rapid verification of model correctness and to guard against use of models that exploit confounding variables. We introduce the challenging new task of explainable multiple abnormality classification in volumetric medical images, in which a model must indicate the regions used to predict each abnormality. To solve this task, we propose a multiple instance learning convolutional neural network, AxialNet, that allows identification of top slices for each abnormality. Next we incorporate HiResCAM, an attention mechanism, to …
Distance Guided Generative Adversarial Network for Explainable Binary Classifications
Macao Polytechnic University（澳门理工大学）；Shanghai Jiao Tong University（上海交通大学）；East China Normal University（华东师范大学）；Zhejiang Sci-Tech University（浙江理工大学）；Hangzhou Normal University（杭州师范大学）
Weakly-supervised segmentation using inherently-explainable classification models and their application to brain tumour classification
Knowledge Engineering Group, Faculty of Computer Science, Otto von Guericke University Magdeburg, Germany；Biomedical Magnetic Resonance, Faculty of Nature Sciences, Otto von Guericke University Magdeburg, Germany；Genomics Research Centre, Human Technopole, Milan, Italy；Institute for Medical Engineering, Faculty of Electrical Engineering；Information Technology, Otto von Guericke University Magdeburg, Germany；Digital Engineering Faculty, University of Potsdam, Germany；Machine Learning, Hasso-Plattner-Institute, Potsdam, Germany；Department of Biomedical Data Science, Stanford University, Stanford, CA, United States；Centre for Behavioural Brain Sciences, Magdeburg, Germany；German Centre for Neurodegenerative Disease, Magdeburg, Germany
