基于加权相关性累积的通用注意力模型可解释性方法
Generic Attention-model Explainability by Weighted Relevance Accumulation
AI总结：
针对现有注意力模型可解释性方法均等累积token相关性易产生失真的问题，提出加权相关性策略，并构建包含多类注意力的CLIPmapper模型开展评估，经多任务扰动测试验证该方法性能优于现有方法。
摘要：
基于注意力的Transformer模型在视觉问答等多模态任务中已取得显著进展。由于注意力方法的可解释性能够通过跨注意力层累积相关性来解释注意力token的内部变化，近年来受到广泛关注。现有方法在注意力过程前后通过均等累积token相关性的方式简单更新相关性，但在相关性累积过程中，不同token值的重要性通常存在差异。本文提出一种加权相关性策略，将token值的重要性纳入考量，以降低均等累积相关性时产生的失真。为评估所提方法，我们提出一个基于CLIP的统一两阶段模型CLIPmapper，通过CLIP编码器及后续的映射器处理视觉-语言任务。CLIPmapper包含自注意力、交叉注意力、单模态注意力和跨模态注意力，因此更适合评估该通用可解释性方法。在视觉问答和图像字幕任务上开展的大量扰动测试验证了，该可解释性方法的性能优于现有方法。
英文摘要：
Attention-based transformer models have achieved remarkable progress in multi-modal tasks, such as visual question answering. The explainability of attention-based methods has recently attracted wide interest as it can explain the inner changes of attention tokens by accumulating relevancy across attention layers. Current methods simply update relevancy by equally accumulating the token relevancy before and after the attention processes. However, the importance of token values is usually different during relevance accumulation. In this paper, we propose a weighted relevancy strategy, which takes the importance of token values into consideration, to reduce distortion when equally accumulating relevance. To evaluate our method, we propose a unified CLIP-based two-stage model, named CLIPmapper, to process Vision-and-Language tasks through CLIP encoder and a following mapper. CLIPmapper consists of self-attention, cross-attention, single-modality, and cross-modality attention, thus it is more suitable for evaluating our generic explainability method. Extensive perturbation tests on visual question answering and image captioning validate that our explainability method outperforms existing methods.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
