arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2302.09304cs.CLcs.AI
收藏

Transformer激活空间分析中的可解释性:一项聚焦综述

Interpretability in Activation Space Analysis of Transformers: A Focused Survey

German Research Center for Artificial Intelligence (DFKI)(德国人工智能研究中心(DFKI))

Soniya Vijayakumar

AI总结:

本文综述transformer前馈层神经元激活所构成激活空间的可解释性方法,梳理有限现有工作并指出未来方向。

摘要:

随着transformer的出现,自然语言处理领域取得了诸多突破。自那时起,它们始终保持最先进水平,同时也出现了大量关于分析、解释和评估注意力层及底层嵌入空间的研究。除自注意力层之外,transformer中的前馈层也是一个重要的架构组成部分。从大量研究中,我们观察到其作用尚未得到充分探索。我们聚焦于被称为激活空间(Activation Space)的潜在空间,该空间由来自这些前馈层的神经元激活构成。在这篇综述论文中,我们回顾了考察该激活空间中所发生学习过程的可解释性方法。由于该方向目前仅有有限研究,我们对每项工作进行了细致考察,并指出了潜在的未来研究方向。我们希望本文能够为加强激活空间分析迈出一步。

英文摘要:

The field of natural language processing has reached breakthroughs with the advent of transformers. They have remained state-of-the-art since then, and there also has been much research in analyzing, interpreting, and evaluating the attention layers and the underlying embedding space. In addition to the self-attention layers, the feed-forward layers in the transformer are a prominent architectural component. From extensive research, we observe that its role is under-explored. We focus on the latent space, known as the Activation Space, that consists of the neuron activations from these feed-forward layers. In this survey paper, we review interpretability methods that examine the learnings that occurred in this activation space. Since there exists only limited research in this direction, we conduct a detailed examination of each work and point out potential future directions of research. We hope our work provides a step towards strengthening activation space analysis.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑