arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2406.11944cs.LGcs.CL
收藏

Transcoders 发现可解释的 LLM 特征电路

Transcoders Find Interpretable LLM Feature Circuits

Yale University(耶鲁大学);Columbia University(哥伦比亚大学)

Jacob Dunefsky, Philippe Chlenski, Neel Nanda

AI总结:

本文提出使用 transcoders 对 MLP 层进行基于权重的电路分析,将模型计算分解为可解释电路,并在多种规模语言模型上验证其有效性,获得对 GPT2-small 中“大于电路”的新见解。

摘要:

机制可解释性的一个关键目标是电路分析:寻找对应于特定行为或能力的模型的稀疏子图。然而,MLP 子层使得对基于 Transformer 的语言模型进行细粒度电路分析变得困难。特别是,可解释的特征——例如通过稀疏自编码器(SAEs)发现的那些特征——通常是极多神经元的线性组合,每个神经元都有其自身的非线性需要考虑。因此,在这种设置下的电路分析要么产生难以处理的庞大电路,要么无法区分局部和全局行为。为了解决这个问题,我们探索了 transcoders,它试图用更宽、稀疏激活的 MLP 层来忠实近似一个密集激活的 MLP 层。我们引入了一种新方法,使用 transcoders 通过 MLP 子层进行基于权重的电路分析。得到的电路可以简洁地分解为输入相关和输入不变项。然后,我们成功地在具有 120M、410M 和 1.4B 参数的语言模型上训练了 transcoders,并发现它们在稀疏性、忠实性和人类可解释性方面至少与 SAEs 相当。最后,我们将 transcoders 应用于逆向工程模型中的未知电路,并获得了关于 GPT2-small 中“大于电路”的新见解。我们的结果表明,transcoders 可以有效地将涉及 MLP 的模型计算分解为可解释的电路。代码可在 https://github.com/jacobdunefsky/transcoder_circuits/ 获取。

英文摘要:

A key goal in mechanistic interpretability is circuit analysis: finding sparse subgraphs of models corresponding to specific behaviors or capabilities. However, MLP sublayers make fine-grained circuit analysis on transformer-based language models difficult. In particular, interpretable features -- such as those found by sparse autoencoders (SAEs) -- are typically linear combinations of extremely many neurons, each with its own nonlinearity to account for. Circuit analysis in this setting thus either yields intractably large circuits or fails to disentangle local and global behavior. To address this we explore transcoders, which seek to faithfully approximate a densely activating MLP layer with a wider, sparsely-activating MLP layer. We introduce a novel method for using transcoders to perform weights-based circuit analysis through MLP sublayers. The resulting circuits neatly factorize into input-dependent and input-invariant terms. We then successfully train transcoders on language models with 120M, 410M, and 1.4B parameters, and find them to perform at least on par with SAEs in terms of sparsity, faithfulness, and human-interpretability. Finally, we apply transcoders to reverse-engineer unknown circuits in the model, and we obtain novel insights regarding the "greater-than circuit" in GPT2-small. Our results suggest that transcoders can prove effective in decomposing model computations involving MLPs into interpretable circuits. Code is available at https://github.com/jacobdunefsky/transcoder_circuits/.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑