Explainable Deep Classification Models for Domain Generalization
Huawei Ireland Research Center（华为爱尔兰研究中心）；Boston University（波士顿大学）；Istituto Italiano di Tecnologia（意大利理工学院）；Naver Labs Europe（欧洲Naver实验室）；Microsoft（微软）；Adobe Research（Adobe研究院）；Università di Verona（维罗纳大学）
Andrea Zunino, Sarah Adel Bargal, Riccardo Volpi, Mehrnoosh Sameki, Jianming Zhang, Stan Sclaroff, Vittorio Murino, Kate Saenko
Conventionally, AI models are thought to trade off explainability for lower accuracy. We develop a training strategy that not only leads to a more explainable AI system for object classification, but as a consequence, suffers no perceptible accuracy degradation. Explanations are defined as regions of visual evidence upon which a deep classification network makes a decision. This is represented in the form of a saliency map conveying how much each pixel contributed to the network's decision. Our training strategy enforces a periodic saliency-based feedback to encourage the model to focus on the image regions that directly correspond to the ground-truth object. We quantify explainability using an automated metric, and using human judgement. We propose explainability as a means for bridging the visual-semantic gap between different domains where model explanations are used as a means of disentagling domain specific information from otherwise relevant features. We demonstrate that this leads to improved generalization to new domains without hindering performance on the original domain.
Deep learning methods exhibit outstanding performance in synthetic aperture radar (SAR) image interpretation tasks. However, these are black box models that limit the comprehension of their predictions. Therefore, to meet this challenge, we have utilized explainable artificial intelligence (XAI) methods for the SAR image classification task. Specifically, we trained state-of-the-art convolutional neural networks for each polarization format on OpenSARUrban dataset and then investigate eight explanation methods to analyze the predictions of the CNN classifiers of SAR images. These XAI methods a…
Institute for Datability Science (IDS)（数据科学研究所）；Graduate School of Medicine Osaka University（大阪大学医学研究生院）
Few-shot learning (FSL) approaches are usually based on an assumption that the pre-trained knowledge can be obtained from base (seen) categories and can be well transferred to novel (unseen) categories. However, there is no guarantee, especially for the latter part. This issue leads to the unknown nature of the inference process in most FSL methods, which hampers its application in some risk-sensitive areas. In this paper, we reveal a new way to perform FSL for image classification, using visual representations from the backbone model and weights generated by a newly-emerged explainable classi…
No Masks Needed: Explainable AI for Deriving Segmentation from Classification
Dept. Electrical and Electronic Engineering, Imperial College London（帝国理工学院伦敦分校电子与电气工程系）；Centre for Vision, Speech and Signal Processing, University of Surrey（萨里大学视觉、语音与信号处理中心）
Training-free Test-time Improvement for Explainable Medical Image Classification
Department of Biomedical Engineering, Peking University, Beijing, China（北京大学生物医学工程系）；Institute of Medical Technology, Peking University, Beijing, China（北京大学医学技术研究所）；National Biomedical Imaging Center, Peking University, Beijing, China（北京大学国家生物医学成像中心）；Beijing Institute of Technology, Beijing, China（北京理工大学）；Equal Contribution（同等贡献）；Corresponding Author（通讯作者）
Northwestern Polytechnical University（西北工业大学）；Remote Sensing Technology Institute (IMF), German Aerospace Center (DLR)（德国航空航天中心（DLR）遥感技术研究所（IMF））；University POLITEHNICA of Bucharest (UPB)（布加勒斯特理工大学（UPB））
Integrating the special electromagnetic characteristics of Synthetic Aperture Radar (SAR) in deep neural networks is essential in order to enhance the explainability and physics awareness of deep learning. In this paper, we first propose a novel physically explainable convolutional neural network for SAR image classification, namely physics guided and injected learning (PGIL). It comprises three parts: (1) explainable models (XM) to provide prior physics knowledge, (2) physics guided network (PGN) to encode the knowledge into physics-aware features, and (3) physics injected network (PIN) to ad…
Minimum Data, Maximum Impact: 20 annotated samples for explainable lung nodule classification
Experimental Radiology, Ulm University Medical Center, Ulm, Germany（实验放射学，乌尔姆大学医学中心，乌尔姆，德国）；Department of Diagnostic and Interventional Radiology, Ulm University Medical Center, Ulm, Germany（诊断与介入放射学系，乌尔姆大学医学中心，乌尔姆，德国）；XAIRAD - Cooperation for Artificial Intelligence in Experimental Radiology, Ulm, Germany（XAIRAD - 实验放射学中的人工智能合作）
Center for Public Health Genomics（公共卫生基因组学中心）；Department of Biomedical Engineering University of Virginia（弗吉尼亚大学生物医学工程系）；University of Virginia（弗吉尼亚大学）
White Blood Cell (WBC) Leukaemia is detected through image-based classification. Convolutional Neural Networks are used to learn the features needed to classify images of cells a malignant or healthy. However, this type of model requires learning a large number of parameters and is difficult to interpret and explain. Explainable AI (XAI) attempts to alleviate this issue by providing insights to how models make decisions. Therefore, we present an XAI model which uses only 24 explainable and interpretable features and is highly competitive to other approaches by outperforming them by about 4.38\…
Learning Interpretable Queries for Explainable Image Classification with Information Pursuit
LMU Munich（慕尼黑大学）；AWS AI Labs（AWS人工智能实验室）；University of Pennsylvania（宾夕法尼亚大学）；Technical University of Munich（慕尼黑技术大学）；Munich Center of Machine Learning（慕尼黑机器学习中心）
The Open University（开放大学）；Microsoft（微软）；Tel-Aviv University（特拉维夫大学）；Bar-Ilan University（巴伊兰大学）；Technion（以色列理工学院）；Tel Aviv University（特拉维夫大学）
We present Gradient Activation Maps (GAM) - a machinery for explaining predictions made by visual similarity and classification models. By gleaning localized gradient and activation information from multiple network layers, GAM offers improved visual explanations, when compared to existing alternatives. The algorithmic advantages of GAM are explained in detail, and validated empirically, where it is shown that GAM outperforms its alternatives across various tasks and datasets.
University of Southern California（南加州大学）；Columbia University（哥伦比亚大学）；Tencent America（腾讯美国）
An explainable machine learning method for point cloud classification, called the PointHop method, is proposed in this work. The PointHop method consists of two stages: 1) local-to-global attribute building through iterative one-hop information exchange, and 2) classification and ensembles. In the attribute building stage, we address the problem of unordered point cloud data using a space partitioning procedure and developing a robust descriptor that characterizes the relationship between a point and its one-hop neighbor in a PointHop unit. When we put multiple PointHop units in cascade, the a…
Technical University of Kaiserslautern（凯泽斯劳滕工业大学）；Max Planck Institute for Informatics（马克斯·普朗克信息学研究所）；Technical University of Berlin（柏林工业大学）；Google Research, Brain Team（谷歌研究院大脑团队）；Korea University（高丽大学）
Deep one-class classification variants for anomaly detection learn a mapping that concentrates nominal samples in feature space causing anomalies to be mapped away. Because this transformation is highly non-linear, finding interpretations poses a significant challenge. In this paper we present an explainable deep one-class classification method, Fully Convolutional Data Description (FCDD), where the mapped samples are themselves also an explanation heatmap. FCDD yields competitive detection performance and provides reasonable explanations on common anomaly detection benchmarks with CIFAR-10 an…
Massachusetts General Hospital（麻省总医院）；Harvard Medical School（哈佛医学院）；Singapore Eye Research Institute（新加坡眼科研究所）；Duke-NUS Medical School（杜克-新加坡国立大学医学院）；Institute for Molecular and Clinical Ophthalmology（分子与临床眼科研究所）；National University of Singapore（新加坡国立大学）
In this paper, we propose an explainable and interpretable diabetic retinopathy (ExplainDR) classification model based on neural-symbolic learning. To gain explainability, a highlevel symbolic representation should be considered in decision making. Specifically, we introduce a human-readable symbolic representation, which follows a taxonomy style of diabetic retinopathy characteristics related to eye health conditions to achieve explainability. We then include humanreadable features obtained from the symbolic representation in the disease prediction. Experimental results on a diabetic retinopa…
Sir Run Run Shaw Hospital（邵逸夫医院）；School of Medicine, Zhejiang University（浙江大学医学院）；School of Computer Science and Technology, Hangzhou Dianzi University（杭州电子科技大学计算机科学与技术学院）；College of Medicine, Zhejiang University（浙江大学医学院）
Automatic pulmonary nodules classification is significant for early diagnosis of lung cancers. Recently, deep learning techniques have enabled remarkable progress in this field. However, these deep models are typically of high computational complexity and work in a black-box manner. To combat these challenges, in this work, we aim to build an efficient and (partially) explainable classification model. Specially, we use \emph{neural architecture search} (NAS) to automatically search 3D network architectures with excellent accuracy/speed trade-off. Besides, we use the convolutional block attenti…
Deep neural networks (DNs) provide superhuman performance in numerous computer vision tasks, yet it remains unclear exactly which of a DN's units contribute to a particular decision. NeuroView is a new family of DN architectures that are interpretable/explainable by design. Each member of the family is derived from a standard DN architecture by vector quantizing the unit output values and feeding them into a global linear classifier. The resulting architecture establishes a direct, causal link between the state of each unit and the classification decision. We validate NeuroView on standard dat…
BRAC University（BRAC大学）；Islamic University of Technology（伊斯兰技术大学）
In the world of medical diagnostics, the adoption of various deep learning techniques is quite common as well as effective, and its statement is equally true when it comes to implementing it into the retina Optical Coherence Tomography (OCT) sector, but (i)These techniques have the black box characteristics that prevent the medical professionals to completely trust the results generated from them (ii)Lack of precision of these methods restricts their implementation in clinical and complex cases (iii)The existing works and models on the OCT classification are substantially large and complicated…
University of New Brunswick（新不伦瑞克大学）；University of Toronto（多伦多大学）
Explaining the generalization characteristics of deep learning is an emerging topic in advanced machine learning. There are several unanswered questions about how learning under stochastic optimization really works and why certain strategies are better than others. In this paper, we address the following question: \textit{can we probe intermediate layers of a deep neural network to identify and quantify the learning quality of each layer?} With this question in mind, we propose new explainability metrics that measure the redundant information in a network's layers using a low-rank factorizatio…
National Technical University of Athens（雅典国家技术大学）；Athena Research Center（雅典娜研究中心）
Although deep neural networks hold the state-of-the-art in several remote sensing tasks, their black-box operation hinders the understanding of their decisions, concealing any bias and other shortcomings in datasets and model performance. To this end, we have applied explainable artificial intelligence (XAI) methods in remote sensing multi-label classification tasks towards producing human-interpretable explanations and improve transparency. In particular, we utilized and trained deep learning models with state-of-the-art performance in the benchmark BigEarthNet and SEN12MS datasets. Ten XAI m…
Neural Cellular Automata for Lightweight, Robust and Explainable Classification of White Blood Cell Images
Institute of AI for Health, Helmholtz Zentrum München – German Research Center for Environmental Health（慕尼黑亥姆霍兹中心人工智能健康研究所（德国环境健康研究中心））；Faculty of Mathematics, Technical University Munich（慕尼黑工业大学数学学院）；Computer Aided Medical Procedures, Technical University of Munich（慕尼黑工业大学计算机辅助医疗程序研究所）；Computer Aided Medical Procedures, Johns Hopkins University（约翰霍普金斯大学计算机辅助医疗程序研究所）；Helmholtz AI, Helmholtz Zentrum München - German Research Center for Environmental Health（亥姆霍兹人工智能中心（慕尼黑亥姆霍兹中心，德国环境健康研究中心））
We propose a method to classify cardiac pathology based on a novel approach to extract image derived features to characterize the shape and motion of the heart. An original semi-supervised learning procedure, which makes efficient use of a large amount of non-segmented images and a small amount of images segmented manually by experts, is developed to generate pixel-wise apparent flow between two time points of a 2D+t cine MRI image sequence. Combining the apparent flow maps and cardiac segmentation masks, we obtain a local apparent flow corresponding to the 2D motion of myocardium and ventricu…
MedicalPatchNet: A Patch-Based Self-Explainable AI Architecture for Chest X-ray Classification
Department of Diagnostic and Interventional Radiology of the University Hospital RWTH Aachen（鲁尔大学亚琛医院诊断与介入放射科）；Else Kroener Fresenius Center for Digital Health, Technical University Dresden（德累斯顿技术大学埃勒克罗纳弗雷斯内尔数字健康中心）；Department of Medicine III, University Hospital RWTH Aachen（鲁尔大学亚琛医院第三医学部）；Pathology & Data Analytics, Leeds Institute of Medical Research at St James’s, University of Leeds（利兹大学圣詹姆斯医院医学研究所以及病理学与数据科学）；Department of Medicine I, University Hospital Dresden（德累斯顿大学第一医学部）；Medical Oncology, National Center for Tumor Diseases (NCT), University Hospital Heidelberg（海德堡大学国家肿瘤疾病中心（NCT）肿瘤医学部）
Evaluating Single and Multi-Omics Based Explainable Artificial Intelligence (MOXAI) for Molecular Subclass Classification of Adult-Type Diffuse Gliomas
Vrije Universiteit Brussel (VUB)（布鲁塞尔自由大学）；Universidad de las Ciencias Informáticas (UCI)（信息科学大学）；Interuniversity Microelectronics Centre (IMEC)（微电子研究中心）
Our motivating application is a real-world problem: COVID-19 classification from CT imaging, for which we present an explainable Deep Learning approach based on a semi-supervised classification pipeline that employs variational autoencoders to extract efficient feature embedding. We have optimized the architecture of two different networks for CT images: (i) a novel conditional variational autoencoder (CVAE) with a specific architecture that integrates the class labels inside the encoder layers and uses side information with shared attention layers for the encoder, which make the most of the c…
Keen Eye（基恩眼科）；Laboratoire Traitement et Communication de l'Information（信息处理与通信实验室）；Telecom ParisTech（巴黎高等电信学院）；Universite Paris-Saclay（巴黎萨克雷大学）
This article adresses the problem of automatic squamous cells classification for cervical cancer screening using Deep Learning methods. We study different architectures on a public dataset called Herlev dataset, which consists in classifying cells, obtained by cervical pap smear, regarding the severity of the abnormalities they represent. Furthermore, we use an attribution method to understand which cytomorphological features are actually learned as discriminative to classify severity of the abnormalities. Through this paper, we show how we trained a performant classifier: 74.5\% accuracy on s…
The complexity of state-of-the-art modeling techniques for image classification impedes the ability to explain model predictions in an interpretable way. Existing explanation methods generally create importance rankings in terms of pixels or pixel groups. However, the resulting explanations lack an optimal size, do not consider feature dependence and are only related to one class. Counterfactual explanation methods are considered promising to explain complex model decisions, since they are associated with a high degree of human interpretability. In this paper, SEDC is introduced as a model-agn…
Fraunhofer Institute for Integrated Circuits IIS（弗劳恩霍夫集成电路研究所IIS）；Nuremberg Institute of Technology Georg Simon Ohm（纽伦堡乔治·西蒙·欧姆技术学院）
Would you trust physicians if they cannot explain their decisions to you? Medical diagnostics using machine learning gained enormously in importance within the last decade. However, without further enhancements many state-of-the-art machine learning methods are not suitable for medical application. The most important reasons are insufficient data set quality and the black-box behavior of machine learning algorithms such as Deep Learning models. Consequently, end-users cannot correct the model's decisions and the corresponding explanations. The latter is crucial for the trustworthiness of machi…
This study explores the use of the Dirichlet Variational Autoencoder (DirVAE) for learning disentangled latent representations of chest X-ray (CXR) images. Our working hypothesis is that distributional sparsity, as facilitated by the Dirichlet prior, will encourage disentangled feature learning for the complex task of multi-label classification of CXR images. The DirVAE is trained using CXR images from the CheXpert database, and the predictive capacity of multi-modal latent representations learned by DirVAE models is investigated through implementation of an auxiliary multi-label classificatio…
Benchmarking Class Activation Map Methods for Explainable Brain Hemorrhage Classification on Hemorica Dataset
Faculty of Computer Engineering, K. N. Toosi University of Technology（计算机工程学院，K.N.托菲大学）；Faculty of Electrical Engineering, K. N. Toosi University of Technology（电气工程学院，K.N.托菲大学）；Faculty of Mechanical Engineering, Tarbiat Modares University（机械工程学院，塔里比亚特莫达res大学）
XMACNet: An Explainable Lightweight Attention based CNN with Multi Modal Fusion for Chili Disease Classification
Vellore Institute of Technology（韦洛尔理工学院）；Angel College of Engineering and Technology（安吉尔工程技术学院）；Vel Tech Rangarajan Dr.Sagunthala R&D Institute of Science and Technology（Vel Tech朗加拉詹·萨贡塔拉博士科技研究院）；Bannari Amman Institute of Technology（班纳里·安曼理工学院）
Pontifical Catholic University of Rio de Janeiro（里约热内卢天主教大学）
A novel evolutionary approach for Explainable Artificial Intelligence is presented: the "Evolved Explanations" model (EvEx). This methodology consists in combining Local Interpretable Model Agnostic Explanations (LIME) with Multi-Objective Genetic Algorithms to allow for automated segmentation parameter tuning in image classification tasks. In this case, the dataset studied is Patch-Camelyon, comprised of patches from pathology whole slide images. A publicly available Convolutional Neural Network (CNN) was trained on this dataset to provide a binary classification for presence/absence of lymph…
Pathologist-Like Explanations Unveiled: an Explainable Deep Learning System for White Blood Cell Classification
Indian Statistical Institute（印度统计研究所）；Drs Tribedi & Roy Diagnostic Laboratory（特里贝迪-罗伊医生诊断实验室）；Zoho Corporation（卓豪公司）；Centre for Digital Healthcare Technologies, MAKAUT（MAKAUT数字医疗技术中心）
Explainable Binary Classification of Separable Shape Ensembles
Applied & Computational Mathematics Div.（应用与计算数学部）；National Institute of Standards & Technology（国家标准与技术研究院）；Dept. of Mathematics and Statistics（数学与统计学系）；Portland State University（波特兰州立大学）；Computational Science Center（计算科学中心）；National Laboratory of the Rockies（落基山脉国家实验室）
