Deep neural networks (DNNs) have greatly impacted numerous fields over the past decade. Yet despite exhibiting superb performance over many problems, their black-box nature still poses a significant challenge with respect to explainability. Indeed, explainable artificial intelligence (XAI) is crucial in several fields, wherein the answer alone -- sans a reasoning of how said answer was derived -- is of little value. This paper uncovers a troubling property of explanation methods for image-based DNNs: by making small visual changes to the input image -- hardly influencing the network's output -- we demonstrate how explanations may be arbitrarily manipulated through the use of evolution strategies. Our novel algorithm, AttaXAI, a model-agnostic, adversarial attack on XAI algorithms, only requires access to the output logits of a classifier and to the explanation map; these weak assumptions render our approach highly useful where real-world models and data are concerned. We compare our method's performance on two benchmark datasets -- CIFAR100 and ImageNet -- using four different pretrained deep-learning models: VGG16-CIFAR100, VGG16-ImageNet, MobileNet-CIFAR100, and Inception-v3-ImageNet. We find that the XAI methods can be manipulated without the use of gradients or other model internals. Our novel algorithm is successfully able to manipulate an image in a manner imperceptible to the human eye, such that the XAI method outputs a specific explanation map. To our knowledge, this is the first such method in a black-box setting, and we believe it has significant value where explainability is desired, required, or legally mandatory.
Designing Counterfactual Generators using Deep Model Inversion
Lawrence Livermore National Laboratory(劳伦斯利弗莫尔国家实验室);Arizona State University(亚利桑那州立大学);IBM Research AI(IBM研究院人工智能部门);Stanford University(斯坦福大学)
解释技术将小的、可解释的变化合成到给定的图像,同时在模型预测中产生所需的变化,这种解释技术已经成为内省黑盒模型的流行方法。通常被称为反事实,合成的解释需要包含可辨别的变化(为了易于解释),同时也是现实的(与数据流形的一致性)。在本文中,我们关注的情况是,我们只能访问经过训练的深度分类器,而不能访问实际的训练数据。虽然已经探索了从训练分布中反演深度模型以合成图像的问题,但我们的目标是开发一种深度反演方法来为给定的查询图像生成反事实解释。尽管它们在条件图像合成中是有效的,但我们表明现有的深度反演方法不足以产生有意义的反事实。我们提出了DISC(Deep Inversion for Synthesizing Counterfactuals),它通过利用(a)更强的图像先验,(b)结合新的流形一致性目标和(c)采用渐进式优化策略来改进深度反演。我们发现,除了产生视觉上有意义的解释外,来自DISC的反事实在学习分类器决策边界方面也很有效,并且对未知的测试时损坏具有鲁棒性。
SIDA: Social Media Image Deepfake Detection, Localization and Explanation with Large Multimodal Model
University of Liverpool, UK(利物浦大学);Nanyang Technological University, SG(南洋理工大学);WMG, University of Warwick(沃里克大学工程学院);The Chinese University of Hong Kong, Shenzhen, Guangdong, China(香港中文大学(深圳))
DeepMiner: Discovering Interpretable Representations for Mammogram Classification and Explanation
Princeton University(普林斯顿大学);The Chinese University of Hong Kong(香港中文大学);Microsoft(微软);UCLA(加州大学洛杉矶分校);Beth Israel Deaconess Medical Center(贝斯以色列女执事医疗中心);Microsoft Research New England(微软新英格兰研究院);VSCO
Visual Explanations from Hadamard Product in Multimodal Deep Networks
Seoul National University(首尔大学)
对模型学习表征的直观解释有助于理解学习的基本原理。以前的作品中的注意力模型用于在图像或文本上使用它们的学习权重来可视化关注区域,以确认它们的预期机制。Kim et al.(2016)表明,多模态深度网络中的Hadamard产品,以视觉问答任务的联合功能而闻名,隐含地执行视觉输入的注意机制。在这项工作中,我们扩展了他们的工作,以表明多模态深度网络中的Hadamard乘积不仅适用于视觉输入,而且还适用于文本输入,同时使用所提出的基于梯度的可视化技术。阿达玛产品的注意力效果可视化视觉和文本输入通过分析两个输入和输出的阿达玛产品与所提出的方法,并与学习的视觉问答模型的注意力权重进行比较。
Solving the enigma: Enhancing faithfulness and comprehensibility in explanations of deep networks
University of Cambridge(剑桥大学);University of Virginia(弗吉尼亚大学);University of Oslo(奥斯陆大学);Østfold Hospital(Østfold医院);Karolinska Institutet(卡罗林斯卡学院);Diakonhjemmet Hospital(迪亚科内姆医院)
DeepOpht: Medical Report Generation for Retinal Images via Deep Models and Visual Explanation
Georgia Institute of Technology(佐治亚理工学院);University of Cambridge(剑桥大学);Bern University Hospital(伯尔尼大学医院);National Taiwan University(台湾大学);University of California, Berkeley(加州大学伯克利分校);Tri-Service General Hospital(三军总医院);Beijing Friendship Hospital(北京友谊医院);King Abdullah University of Science and Technology (KAUST)(阿卜杜拉国王科技大学);University of Amsterdam(阿姆斯特丹大学)
OPTIMUS-Prime: Minimal and Sufficient Concept Explanations for Deep Vision Models
Université de Lorraine(洛林大学);CentraleSupélec Loria(中央超导Loria);CNRS(国家科学研究中心);Metz, France(法国梅斯);Université de technologie de Compiègne UMR CNRS 7253 Heudiasyc(图卢兹技术大学UMR CNRS 7253 Heudiasyc);France(法国)
Generating visual explanations from deep networks using implicit neural representations
Institute of Fundamental Technological Research, Polish Academy of Sciences(波兰科学院基础技术研究所);RIKEN Center for Brain Science(理化学研究所脑科学中心);Samsung AI Center Warsaw(三星人工智能中心华沙分部)
Visual explanations for polyp detection: How medical doctors assess intrinsic versus extrinsic explanations
SimulaMet(西穆拉大都会研究所);UIT The Artic University of Norway(挪威UIT北极大学);Sahlgrenska University Hospital-Mölndal(萨尔格伦斯卡大学医院-默恩达尔);Swedish Medical Group(瑞典医疗集团);Oslo Metropolitan University(奥斯陆城市大学);Norwegian University of Science and Technology(挪威科技大学)
Pathologist-Like Explanations Unveiled: an Explainable Deep Learning System for White Blood Cell Classification
Indian Statistical Institute(印度统计研究所);Drs Tribedi & Roy Diagnostic Laboratory(特里贝迪-罗伊医生诊断实验室);Zoho Corporation(卓豪公司);Centre for Digital Healthcare Technologies, MAKAUT(MAKAUT数字医疗技术中心)
Methodological Explainability Evaluation of an Interpretable Deep Learning Model for Post-Hepatectomy Liver Failure Prediction Incorporating Counterfactual Explanations and Layerwise Relevance Propagation: A Prospective In Silico Trial
Maastricht University(马斯特里赫特大学);Maastricht University Medical Center+(马斯特里赫特大学医学中心+);The First Affiliated Hospital of Sun Yat-sen University(中山大学附属第一医院);Guizhou University(贵州大学);Université Libre de Bruxelles(布鲁塞尔自由大学)
Looking in the mirror: A faithful counterfactual explanation method for interpreting deep image classification models
Australian Institute for Machine Learning, University of Adelaide, Australia(澳大利亚机器学习研究所,阿德莱德大学,澳大利亚);Deakin University, Australia(德肯大学,澳大利亚);Flinders University, Australia(弗林德斯大学,澳大利亚)