基于Shapley解释方法的注意与非注意
Attention vs non-attention for a Shapley-based explanation method
摘要：
可解释人工智能领域最近出现了高度非线性深度神经网络的解释方法数量的爆炸式增长。这种方法--通常在计算机视觉领域提出和测试--在多大程度上适合于解决NLP中的可解释性挑战，还相对未被探索。在这项工作中，我们考虑上下文分解（CD）-一种基于Shapley的输入特征归因方法，已被证明适用于经常性NLP模型-我们测试了它对包含注意力操作的模型的有用程度。为此，我们扩展CD，以涵盖基于注意力的模型所需的操作。然后，我们比较长距离主谓关系的处理模型和不注意，考虑到一些不同的句法结构，在两种不同的语言：英语和荷兰语。我们的实验证实，CD也可以成功地应用于基于注意力的模型，为现代神经网络提供了一种替代的基于Shapley的归因方法。特别是，使用CD，我们表明，英语和荷兰语模型表现出类似的处理行为，但在引擎盖下，我们的注意力和非注意力模型之间存在一致的差异。
英文摘要：
The field of explainable AI has recently seen an explosion in the number of explanation methods for highly non-linear deep neural networks. The extent to which such methods -- that are often proposed and tested in the domain of computer vision -- are appropriate to address the explainability challenges in NLP is yet relatively unexplored. In this work, we consider Contextual Decomposition (CD) -- a Shapley-based input feature attribution method that has been shown to work well for recurrent NLP models -- and we test the extent to which it is useful for models that contain attention operations. To this end, we extend CD to cover the operations necessary for attention-based models. We then compare how long distance subject-verb relationships are processed by models with and without attention, considering a number of different syntactic structures in two different languages: English and Dutch. Our experiments confirm that CD can successfully be applied for attention-based models as well, providing an alternative Shapley-based attribution method for modern neural networks. In particular, using CD, we show that the English and Dutch models demonstrate similar processing behaviour, but that under the hood there are consistent differences between our attention and non-attention models.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
