注意力不能成为解释
Attention cannot be an Explanation
摘要：
基于注意力的解释（即显着性图），通过提供可解释性的黑盒模型，如深神经网络，被认为是提高人类的信任和依赖的基础模型。最近，它已被证明，注意力的权重往往是不相关的基于梯度的措施的功能的重要性。受此启发，我们提出了一个后续问题：“假设我们只考虑注意力权重与特征重要性密切相关的任务，那么这些基于注意力的解释在增加人类对底层模型的信任和依赖方面有多有效？".换句话说，我们能用注意力来解释吗？我们进行了广泛的人类研究实验，旨在定性和定量地评估基于注意力的解释在增加人类信任和依赖方面的适用程度。我们的实验结果表明，注意力不能作为一种解释。
英文摘要：
Attention based explanations (viz. saliency maps), by providing interpretability to black box models such as deep neural networks, are assumed to improve human trust and reliance in the underlying models. Recently, it has been shown that attention weights are frequently uncorrelated with gradient-based measures of feature importance. Motivated by this, we ask a follow-up question: "Assuming that we only consider the tasks where attention weights correlate well with feature importance, how effective are these attention based explanations in increasing human trust and reliance in the underlying models?". In other words, can we use attention as an explanation? We perform extensive human study experiments that aim to qualitatively and quantitatively assess the degree to which attention based explanations are suitable in increasing human trust and reliance. Our experiment results show that attention cannot be used as an explanation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
