arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2301.05062cs.LGcs.AIstat.ML
收藏

Tracr:将Transformer编译为可解释性研究实验室

Tracr: Compiled Transformers as a Laboratory for Interpretability

Google DeepMind(谷歌DeepMind)

David Lindner, János Kramár, Sebastian Farquhar, Matthew Rahtz, Thomas McGrath, Vladimir Mikulik

AI总结:

本研究提出可将人类可读程序编译为仅解码器Transformer的编译器Tracr,其生成的结构已知模型可用于可解释性研究,还能作为评估基准,已开源并通过多类程序验证有效性。

摘要:

我们展示了如何将人类可读的程序“编译”为标准的仅解码器Transformer模型。我们的编译器Tracr会生成结构已知的模型,这种结构可用于设计实验。例如,我们用它来研究执行多步算法的Transformer中的“叠加(superposition)”现象。此外,Tracr编译模型的已知结构可作为评估可解释性方法的基准真值。通常,由于Transformer学习到的“程序”是未知的,我们无法确定一种解释方法是否成功。我们通过实现并检验包括词元频率计算、排序和括号检查在内的程序,验证了我们的方法。我们在https://github.com/google-deepmind/tracr提供了Tracr的开源实现。

英文摘要:

We show how to "compile" human-readable programs into standard decoder-only transformer models. Our compiler, Tracr, generates models with known structure. This structure can be used to design experiments. For example, we use it to study "superposition" in transformers that execute multi-step algorithms. Additionally, the known structure of Tracr-compiled models can serve as ground-truth for evaluating interpretability methods. Commonly, because the "programs" learned by transformers are unknown it is unclear whether an interpretation succeeded. We demonstrate our approach by implementing and examining programs including computing token frequencies, sorting, and parenthesis checking. We provide an open-source implementation of Tracr at https://github.com/google-deepmind/tracr.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑