解释深度图像分类器的重要一步在于识别图像中对模型输出中各个类别分数有贡献的图像区域。然而，由于此类网络的黑盒性质，准确做到这一点是一项困难的任务。大多数现有方法要么使用激活值和梯度，要么通过反复扰动输入来寻找此类归因。我们转而通过训练第二个深度网络（即 Explainer）来解决这一挑战，以预测预训练黑盒分类器（即 Explanandum）的归因。这些归因以掩码的形式提供，仅显示图像中与分类器相关的部分，将其余部分掩盖。与其他方法生成的显著性图相比，我们的方法产生了更清晰、边界更精确的掩码。此外，与大多数现有方法不同，我们的方法能够在单次前向传播中直接生成非常独特的特定类别掩码。这使得所提出的方法在推理期间非常高效。我们表明，在 PASCAL VOC-2007 和 Microsoft COCO-2014 数据集上，我们的归因在视觉和定量方面均优于现有方法。
英文摘要：
An important step towards explaining deep image classifiers lies in the identification of image regions that contribute to individual class scores in the model's output. However, doing this accurately is a difficult task due to the black-box nature of such networks. Most existing approaches find such attributions either using activations and gradients or by repeatedly perturbing the input. We instead address this challenge by training a second deep network, the Explainer, to predict attributions for a pre-trained black-box classifier, the Explanandum. These attributions are provided in the form of masks that only show the classifier-relevant parts of an image, masking out the rest. Our approach produces sharper and more boundary-precise masks when compared to the saliency maps generated by other methods. Moreover, unlike most existing approaches, ours is capable of directly generating very distinct class-specific masks in a single forward pass. This makes the proposed method very efficient during inference. We show that our attributions are superior to established methods both visually and quantitatively with respect to the PASCAL VOC-2007 and Microsoft COCO-2014 datasets.
Institute of Computing Technology, CAS（中国科学院计算技术研究所）；University of Chinese Academy of Sciences（中国科学院大学）；Renmin University of China（中国人民大学）
With the rapid progress of generation technology, it has become necessary to attribute the origin of fake images. Existing works on fake image attribution perform multi-class classification on several Generative Adversarial Network (GAN) models and obtain high accuracies. While encouraging, these works are restricted to model-level attribution, only capable of handling images generated by seen models with a specific seed, loss and dataset, which is limited in real-world scenarios when fake images may be generated by privately trained models. This motivates us to ask whether it is possible to a…
Auburn University（奥本大学）；University of Illinois at Chicago（伊利诺伊大学芝加哥分校）
Attribution methods can provide powerful insights into the reasons for a classifier's decision. We argue that a key desideratum of an explanation method is its robustness to input hyperparameters which are often randomly set or empirically tuned. High sensitivity to arbitrary hyperparameter choices does not only impede reproducibility but also questions the correctness of an explanation and impairs the trust of end-users. In this paper, we provide a thorough empirical study on the sensitivity of existing attribution methods. We found an alarming trend that many methods are highly sensitive to …
The advancements in deep learning-based methods for visual perception tasks have seen astounding growth in the last decade, with widespread adoption in a plethora of application areas from autonomous driving to clinical decision support systems. Despite their impressive performance, these deep learning-based models remain fairly opaque in their decision-making process, making their deployment in human-critical tasks a risky endeavor. This in turn makes understanding the decisions made by these models crucial for their reliable deployment. Explainable AI (XAI) methods attempt to address this by…
Hide-and-Seek Attribution: Weakly Supervised Segmentation of Vertebral Metastases in CT
Institute of Neuroradiology, TUM University Hospital, School of Medicine and Health, Technical University of Munich (TUM)（慕尼黑工业大学大学医院神经放射学研究所，医学与健康学院，慕尼黑工业大学）；Chair for AI in Healthcare and Medicine, TUM and TUM University Hospital（慕尼黑工业大学及大学医院医疗与医学人工智能教席）；Munich Center for Machine Learning (MCML)（慕尼黑机器学习中心）；Institute of Diagnostic and Interventional Radiology, TUM University Hospital, School of Medicine and Health, Technical University of Munich (TUM)（慕尼黑工业大学大学医院诊断与介入放射学研究所，医学与健康学院，慕尼黑工业大学）；Department of Neuroradiology, University Hospital Munich (LMU)（慕尼黑大学医院神经放射学系）；Department of Quantitative Biomedicine, University of Zurich（苏黎世大学定量生物医学系）；Department of Computing, Imperial College London（伦敦帝国理工学院计算系）；Department of Diagnostic and Interventional Radiology, University Hospital Frankfurt（法兰克福大学医院诊断与介入放射学系）；Department of Orthopedics and Sports Orthopedics, TUM University Hospital, School of Medicine and Health, Technical University of Munich (TUM)（慕尼黑工业大学大学医院骨科与运动骨科，医学与健康学院，慕尼黑工业大学）
Recent advances in Generative Adversarial Networks (GANs) have shown increasing success in generating photorealistic images. But they also raise challenges to visual forensics and model attribution. We present the first study of learning GAN fingerprints towards image attribution and using them to classify an image as real or GAN-generated. For GAN-generated images, we further identify their sources. Our experiments show that (1) GANs carry distinct model fingerprints and leave stable fingerprints in their generated images, which support image attribution; (2) even minor differences in GAN tra…
Seoul National University（首尔大学）；ASRI（先进技术研究中心）；INMC（纳米材料与技术研究所）；ISRC（智能系统研究中心）；Institute of Engineering Research（工程研究院）
Weakly supervised segmentation methods using bounding box annotations focus on obtaining a pixel-level mask from each box containing an object. Existing methods typically depend on a class-agnostic mask generator, which operates on the low-level information intrinsic to an image. In this work, we utilize higher-level information from the behavior of a trained object detector, by seeking the smallest areas of the image from which the object detector produces almost the same result as it does from the whole image. These areas constitute a bounding-box attribution map (BBAM), which identifies the…
Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）；University of Chinese Academy of Sciences（中国科学院大学）
Despite the remarkable progress in generative technology, the Janus-faced issues of intellectual property protection and malicious content supervision have arisen. Efforts have been paid to manage synthetic images by attributing them to a set of potential source models. However, the closed-set classification setting limits the application in real-world scenarios for handling contents generated by arbitrary models. In this study, we focus on a challenging task, namely Open-Set Model Attribution (OSMA), to simultaneously attribute images to known models and identify those from unknown ones. Comp…
On Thin Ice: Towards Explainable Conservation Monitoring via Attribution and Perturbations
Duke University（杜克大学）；University of Agder（阿格德大学）；University of Cambridge（剑桥大学）；U.S. National Park Service Department of Interior（美国国家公园管理局）；Alaska Spatial Science Fairbanks, AK（阿拉斯加空间科学研究所）
Institute of AI for Health, Helmholtz Munich – German Research Center for Environmental Health（亥姆霍兹慕尼黑中心——德国环境健康研究中心健康人工智能研究所）；Technical University of Munich（慕尼黑工业大学）；Johns Hopkins University（约翰斯·霍普金斯大学）
Explainability is a key requirement for computer-aided diagnosis systems in clinical decision-making. Multiple instance learning with attention pooling provides instance-level explainability, however for many clinical applications a deeper, pixel-level explanation is desirable, but missing so far. In this work, we investigate the use of four attribution methods to explain a multiple instance learning models: GradCAM, Layer-Wise Relevance Propagation (LRP), Information Bottleneck Attribution (IBA), and InputIBA. With this collection of methods, we can derive pixel-level explanations on for the …
University of Maryland（马里兰大学）；Max Planck Institute for Informatics（马克斯·普朗克信息学研究所）；CISPA Helmholtz Center for Information Security（CISPA亥姆霍兹信息安全中心）
Photorealistic image generation has reached a new level of quality due to the breakthroughs of generative adversarial networks (GANs). Yet, the dark side of such deepfakes, the malicious use of generated media, raises concerns about visual misinformation. While existing research work on deepfake detection demonstrates high accuracy, it is subject to advances in generation techniques and adversarial iterations on detection countermeasure techniques. Thus, we seek a proactive and sustainable solution on deepfake detection, that is agnostic to the evolution of generative models, by introducing ar…
Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）；School of Computer Science, Faculty of Engineering, University of Sydney（悉尼大学工程学院计算机科学学院）
Recently, both long-tailed recognition and object tracking have made great advances individually. TAO benchmark presented a mixture of the two, long-tailed object tracking, in order to further reflect the aspect of the real-world. To date, existing solutions have adopted detectors showing robustness in long-tailed distributions, which derive per-frame results. Then, they used tracking algorithms that combine the temporally independent detections to finalize tracklets. However, as the approaches did not take temporal changes in scenes into account, inconsistent classification results in videos …
Localising Shortcut Learning in Pixel Space via Ordinal Scoring Correlations for Attribution Representations (OSCAR)
School of Biomedical Engineering and Imaging Sciences, King’s College London（伦敦国王学院生物医学工程与成像科学系）；Department of Computer Science, University of Warwick（沃里克大学计算机科学系）
