arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2307.09458cs.LG
收藏

电路分析可解释性是否具备可扩展性?来自Chinchilla多项选择能力的证据

Does Circuit Analysis Interpretability Scale? Evidence from Multiple Choice Capabilities in Chinchilla

Google DeepMind(谷歌DeepMind)

Tom Lieberum, Matthew Rahtz, János Kramár, Neel Nanda, Geoffrey Irving, Rohin Shah, Vladimir Mikulik

AI总结:

该研究针对70B参数Chinchilla模型开展电路分析可扩展性测试,验证了三类现有分析技术可扩展至大模型,同时发现“正确字母”注意力头的特征解释仅具部分有效性,仍待深入探究。

摘要:

电路分析是一种用于理解语言模型内部机制的颇具前景的技术。然而,现有分析均在远非当前最优水平的小型模型上开展。为解决这一问题,我们针对70B参数的Chinchilla模型开展了电路分析案例研究,旨在测试电路分析的可扩展性。具体而言,我们研究了多项选择题问答任务,并探究Chinchilla在已知正确答案文本的情况下识别正确答案标签的能力。我们发现,logit归因(对数概率归因)、注意力模式可视化以及激活修补等现有技术可自然扩展至Chinchilla模型,使我们能够识别并归类一小部分“输出节点”(注意力头与多层感知机MLP)。\n 我们进一步研究了“正确字母”类别的注意力头,旨在理解其特征的语义,结果喜忧参半。对于常规的多项选择题答案,我们在处理多项选择题的答案标签时,在不损失性能的前提下大幅压缩了该注意力头的查询、键和值子空间;同时我们表明,查询和键子空间至少在一定程度上表征了“枚举中的第N项”特征。然而,当我们尝试利用这一解释来理解该注意力头在包含随机化答案标签的更通用分布上的行为时,发现它仅能提供部分解释,这表明关于“正确字母”注意力头在多项选择题问答中的运作机制,仍有更多内容有待研究。

英文摘要:

\emph{Circuit analysis} is a promising technique for understanding the internal mechanisms of language models. However, existing analyses are done in small models far from the state of the art. To address this, we present a case study of circuit analysis in the 70B Chinchilla model, aiming to test the scalability of circuit analysis. In particular, we study multiple-choice question answering, and investigate Chinchilla's capability to identify the correct answer \emph{label} given knowledge of the correct answer \emph{text}. We find that the existing techniques of logit attribution, attention pattern visualization, and activation patching naturally scale to Chinchilla, allowing us to identify and categorize a small set of `output nodes' (attention heads and MLPs). We further study the `correct letter' category of attention heads aiming to understand the semantics of their features, with mixed results. For normal multiple-choice question answers, we significantly compress the query, key and value subspaces of the head without loss of performance when operating on the answer labels for multiple-choice questions, and we show that the query and key subspaces represent an `Nth item in an enumeration' feature to at least some extent. However, when we attempt to use this explanation to understand the heads' behaviour on a more general distribution including randomized answer labels, we find that it is only a partial explanation, suggesting there is more to learn about the operation of `correct letter' heads on multiple choice question answering.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑