面向决策的注意力传播用于视觉Transformer可解释性
Decision-Aware Attention Propagation for Vision Transformer Explainability
AI总结：
本文提出DAP方法，通过注入决策相关的先验知识到Transformer注意力传播中，生成更敏感、紧凑且忠实的属性图，提升ViT的可解释性。
摘要：
视觉Transformer（ViTs）已成为计算机视觉中的主导架构，但其预测过程难以解释，因为信息通过复杂交互在层和注意力头之间传播。现有的基于注意力的解释方法提供了一种直观的方式追踪信息流，但它们主要依赖于原始注意力权重，这些权重不明确反映最终决策，常导致解释的类可区分性有限。相比之下，基于梯度的定位方法在突出类特定证据上更有效，但并未充分利用Transformer的层次注意力传播机制。为了解决这一限制，我们提出决策感知注意力传播（DAP），一种将决策相关的先验注入到Transformer注意力传播中的属性方法。通过通过基于梯度的定位估计token重要性，并将其整合到层级注意力滚动中，该方法捕捉了注意力的结构流动和最相关的最终预测证据。因此，DAP生成的属性图比传统基于注意力的方法更敏感、紧凑和忠实。在不同模型规模的视觉Transformer变体上的广泛实验表明，DAP在定量指标和定性可视化上都一致优于现有基线，表明决策感知传播是提高ViT可解释性的有效方向。
英文摘要：
Vision Transformers (ViTs) have become a dominant architecture in computer vision, yet their prediction process remains difficult to interpret because information is propagated through complex interactions across layers and attention heads. Existing attention based explanation methods provide an intuitive way to trace information flow. However, they rely mainly on raw attention weights, which do not explicitly reflect the final decision and often lead to explanations with limited class discriminability. In contrast, gradient based localization methods are more effective at highlighting class specific evidence, but they do not fully exploit the hierarchical attention propagation mechanism of transformers. To address this limitation, we propose Decision-Aware Attention Propagation (DAP), an attribution method that injects decision-relevant priors into transformer attention propagation. By estimating token importance through gradient based localization and integrating it into layer wise attention rollout, the method captures both the structural flow of attention and the evidence most relevant to the final prediction. Consequently, DAP produces attribution maps that are more class sensitive, compact, and faithful than those generated by conventional attention based methods. Extensive experiments across Vision Transformer variants of different model scales show that DAP consistently outperforms existing baselines in both quantitative metrics and qualitative visualizations, indicating that decision aware propagation is an effective direction for improving ViT interpretability.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
