关于 Vision Transformer 解释的忠实性
On the Faithfulness of Vision Transformer Explanations
AI总结：
针对Vision Transformer事后解释的忠实性评估问题，提出新型评估指标SaCo，通过像素组间显著性分数差异聚合准确度量解释忠实性，并证明梯度与多层聚合能显著增强基于注意力的解释的忠实性。
摘要：
为了解释 Vision Transformer，事后解释方法为输入像素分配显著性分数，提供人类可理解的热力图。然而，这些解释是否反映了模型输出背后的真实依据仍未被充分研究。为了填补这一空白，我们研究了解释的忠实性准则：分配的显著性分数应代表相应输入像素对模型预测的影响。为了评估忠实性，我们引入了 Salience-guided Faithfulness Coefficient (SaCo)，这是一种利用显著性分布关键信息的新型评估指标。具体而言，我们在不同的像素组之间进行成对比较，然后聚合它们显著性分数的差异，从而得到一个表示解释忠实性程度的系数。我们的探索表明，当前的指标难以区分高级解释方法和 Random Attribution，因此无法捕捉忠实性属性。相比之下，我们提出的 SaCo 提供了可靠的忠实性度量，为解释建立了稳健的指标。此外，我们的 SaCo 表明，使用梯度和多层聚合可以显著增强基于注意力的解释的忠实性，为推进 Vision Transformer 可解释性指明了潜在路径。
英文摘要：
To interpret Vision Transformers, post-hoc explanations assign salience scores to input pixels, providing human-understandable heatmaps. However, whether these interpretations reflect true rationales behind the model's output is still underexplored. To address this gap, we study the faithfulness criterion of explanations: the assigned salience scores should represent the influence of the corresponding input pixels on the model's predictions. To evaluate faithfulness, we introduce Salience-guided Faithfulness Coefficient (SaCo), a novel evaluation metric leveraging essential information of salience distribution. Specifically, we conduct pair-wise comparisons among distinct pixel groups and then aggregate the differences in their salience scores, resulting in a coefficient that indicates the explanation's degree of faithfulness. Our explorations reveal that current metrics struggle to differentiate between advanced explanation methods and Random Attribution, thereby failing to capture the faithfulness property. In contrast, our proposed SaCo offers a reliable faithfulness measurement, establishing a robust metric for interpretations. Furthermore, our SaCo demonstrates that the use of gradient and multi-layer aggregation can markedly enhance the faithfulness of attention-based explanation, shedding light on potential paths for advancing Vision Transformer explainability.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
