可解释性是否可迁移？面向视觉Transformer与卷积神经网络的归因方法可控基准测试

中文摘要 AI 辅助

目前关于可解释人工智能（XAI）归因方法有效性的多数证据基于卷积神经网络（CNNs）得出，而针对这些结论是否能推广到如今在计算机视觉领域占主导地位的各类视觉Transformer（ViT）架构的研究十分有限。本文提出一项可控基准测试，从保真度、定位能力、鲁棒性、复杂度和计算成本五个维度评估归因质量。采用标准化框架，对四个算法家族的13种归因方法进行评估，涉及8种代表性骨干网络，涵盖CNNs、各向同性ViTs、层次化Transformer、混合架构及线性注意力Transformer。结果表明，归因性能具有强烈的架构依赖性，在CNNs上确立的排名无法可靠迁移到基于Transformer的模型。基于CAM的方法在CNNs和多数ViTs上，基于传统边界框定位指标的得分最高，但在线性注意力架构上表现不佳；像素级密集掩码评估进一步显示，这些优势很大程度上反映了指标饱和而非准确的定位能力。基于CAM的方法在全局注意力Transformer上还表现出有限的鲁棒性，而注意力rollout方法提供的解释始终稳定，但定位能力较差。此外，保真度相关性对归因方法的区分能力有限，凸显了单一指标评估的局限性。这些发现对现有归因性能的主流结论提出挑战，证明需要采用架构感知的多维度评估；评估框架和基准结果的开源代码可在该httpsURL获取。

英文摘要

Most evidence on the effectiveness of explainable artificial intelligence (XAI) attribution methods has been established on convolutional neural networks (CNNs), with limited investigation into whether these conclusions generalize to the diverse Vision Transformer (ViT) architectures that now dominate computer vision. This paper presents a controlled benchmark that evaluates attribution quality across five dimensions: faithfulness, localization, robustness, complexity, and computational cost. A standardized framework assesses 13 attribution methods from four algorithmic families on eight representative backbones spanning CNNs, isotropic ViTs, hierarchical transformers, hybrid architectures, and linear-attention transformers. The results show that attribution performance is strongly architecture-dependent and that rankings established on CNNs do not reliably transfer to transformer-based models. CAM-based methods achieve the highest scores under the conventional bounding-box localization metric on CNNs and most ViTs but perform poorly on linear-attention architectures. Pixel-level dense-mask evaluation further reveals that these gains largely reflect metric saturation rather than accurate localization. CAM-based methods also exhibit limited robustness on global-attention transformers, whereas attention rollout provides consistently stable explanations with poor localization. Furthermore, faithfulness correlation offers limited discrimination between attribution methods, highlighting the limitations of single-metric evaluation. These findings challenge prevailing conclusions on attribution performance and demonstrate the need for architecture-aware, multi-dimensional evaluation. The open-source code for the evaluation framework and benchmark results is available at https://github.com/Nishan-Charlie/VIT_XAI_Bench.

发表机构

• Faculty of Computing, Sabaragamuwa University of Sri Lanka（斯里兰卡萨巴拉加穆瓦大学计算机学院）
• Faculty of Engineering, University of Moratuwa（莫拉图瓦大学工程学院）
• School of Computing Technologies, RMIT University（RMIT大学计算技术学院）
• School of Engineering & Digital Technologies, University of Southern Queensland（南昆士兰大学工程与数字技术学院）
• School of Engineering & Technologies, UNSW（新南威尔士大学工程与技术学院）
• Faculty of Science and Technology, Charles Darwin University（查尔斯达尔文大学科学技术学院）

机构由 AI 辅助整理，请以论文原文为准。