arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2004.11207cs.CL
收藏

Self-Attention Attribution:解读Transformer内部的信息交互

Self-Attention Attribution: Interpreting Information Interactions Inside Transformer

Microsoft Research(微软研究院)

Yaru Hao, Li Dong, Furu Wei, Ke Xu

摘要:

基于transformer的模型的巨大成功得益于强大的多头自我注意力机制,它学习令牌依赖性并从输入中编码上下文信息。先前的工作努力将模型决策归因于具有不同显着性度量的单个输入特征,但它们未能解释这些输入特征如何相互作用以达到预测。本文提出了一种自我注意归因方法来解释Transformer内部的信息交互。我们以BERT为例进行了广泛的研究。首先,我们应用自我注意归因来识别重要的注意头,而其他人可以修剪边际性能下降。此外,我们提取每个层中最显著的依赖关系来构建属性树,从而揭示了Transformer内部的分层交互。最后,我们表明,归因结果可以用作对抗模式,实现非针对性攻击BERT。

英文摘要:

The great success of Transformer-based models benefits from the powerful multi-head self-attention mechanism, which learns token dependencies and encodes contextual information from the input. Prior work strives to attribute model decisions to individual input features with different saliency measures, but they fail to explain how these input features interact with each other to reach predictions. In this paper, we propose a self-attention attribution method to interpret the information interactions inside Transformer. We take BERT as an example to conduct extensive studies. Firstly, we apply self-attention attribution to identify the important attention heads, while others can be pruned with marginal performance degradation. Furthermore, we extract the most salient dependencies in each layer to construct an attribution tree, which reveals the hierarchical interactions inside Transformer. Finally, we show that the attribution results can be used as adversarial patterns to implement non-targeted attacks towards BERT.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑