arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2403.04523cs.CVcs.AIcs.LGcs.MM
收藏

T-TAME:用于解释卷积网络与视觉Transformer的可训练注意力机制

T-TAME: Trainable Attention Mechanism for Explaining Convolutional Networks and Vision Transformers

Centre for Research and Technology Hellas (CERTH) / Information Technologies Institute (ITI)(希腊研究中心与技术研究中心(CERTH)/信息科技研究所(ITI))

Mariano V. Ntrougkas, Nikolaos Gkalelis, Vasileios Mezaris

AI总结:

本文提出T-TAME,一种兼容Transformer的可训练注意力解释方法,能以单次前向传播生成解释图,并在VGG-16、ResNet-50和ViT-B-16上取得优于现有方法的SOTA可解释性表现。

摘要:

视觉Transformer及其他深度学习架构在图像分类任务中的发展和应用十分迅速。然而,神经网络的“黑箱”特性阻碍了其在可解释性至关重要的应用中的采用。虽然已经提出了一些生成解释的技术,主要针对卷积神经网络,但将此类技术适配到视觉Transformer这一新范式并非易事。本文提出了T-TAME,即面向解释的Transformer兼容可训练注意力机制,这是一种用于解释图像分类任务中深度神经网络的通用方法。所提出的架构和训练技术采用简化的训练流程,可轻松应用于任何卷积或类视觉Transformer神经网络。训练完成后,解释图可在单次前向传播中计算得到;这些解释图与计算代价高昂的基于扰动的可解释性技术的输出相当或更优,达到了SOTA性能。我们将T-TAME应用于三种流行的深度学习分类器架构——VGG-16、ResNet-50和ViT-B-16,并在ImageNet数据集上训练,证明了其相对于现有最先进可解释性方法的改进。对结果的详细分析和消融研究揭示了T-TAME设计选择如何影响所生成解释图的质量。

英文摘要:

The development and adoption of Vision Transformers and other deep-learning architectures for image classification tasks has been rapid. However, the "black box" nature of neural networks is a barrier to adoption in applications where explainability is essential. While some techniques for generating explanations have been proposed, primarily for Convolutional Neural Networks, adapting such techniques to the new paradigm of Vision Transformers is non-trivial. This paper presents T-TAME, Transformer-compatible Trainable Attention Mechanism for Explanations, a general methodology for explaining deep neural networks used in image classification tasks. The proposed architecture and training technique can be easily applied to any convolutional or Vision Transformer-like neural network, using a streamlined training approach. After training, explanation maps can be computed in a single forward pass; these explanation maps are comparable to or outperform the outputs of computationally expensive perturbation-based explainability techniques, achieving SOTA performance. We apply T-TAME to three popular deep learning classifier architectures, VGG-16, ResNet-50, and ViT-B-16, trained on the ImageNet dataset, and we demonstrate improvements over existing state-of-the-art explainability methods. A detailed analysis of the results and an ablation study provide insights into how the T-TAME design choices affect the quality of the generated explanation maps.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑