使用上下文分解在Transformer中高效自动发现电路
Efficient Automated Circuit Discovery in Transformers using Contextual Decomposition
AI总结：
提出CD-T方法，通过递归计算模型计算图节点贡献并剪枝，高效构建大型语言模型中细粒度至注意力头的可解释电路，在标准数据集上以秒级运行时间优于基线并完美复制模型行为。
摘要：
自动化机制可解释性研究因其能将神经网络内部机制的解释扩展到大型模型而备受关注。现有的自动电路发现工作依赖激活修补或其近似方法来识别模型中针对特定任务的子图（电路）。它们通常面临运行缓慢、近似误差以及对特定指标要求（如非零梯度）等问题。在本研究中，我们提出了面向Transformer的上下文分解（CD-T），以在大型语言模型中构建可解释的电路。CD-T能够生成任意抽象层级的电路，并且是首个能够高效生成特定序列位置上细粒度至注意力头级别的电路的方法。CD-T包含一组用于分离模型特征贡献的数学方程。通过使用CD-T递归计算模型计算图中所有节点的贡献并进行剪枝，与最先进的基线方法相比，我们能够将电路发现的运行时间从数小时缩短至数秒。在三个标准电路评估数据集（间接宾语识别、大于比较和文档字符串补全）上，我们证明了CD-T优于ACDC和EAP，能更好地恢复人工电路，在低运行时间下平均达到97%的ROC AUC。此外，我们通过证明我们的电路比高达原始模型规模60%的随机电路的忠实度高80%，提供了CD-T电路的忠实度并非出于随机概率的证据。最后，我们表明在所有任务中，CD-T电路能够使用比基线更少的节点完美复制原始模型的行为（忠实度=1）。我们的结果突显了CD-T在高效自动化机制可解释性方面的巨大潜力，为深入了解大型语言模型的工作机制铺平了道路。
英文摘要：
Automated mechanistic interpretation research has attracted great interest due to its potential to scale explanations of neural network internals to large models. Existing automated circuit discovery work relies on activation patching or its approximations to identify subgraphs in models for specific tasks (circuits). They often suffer from slow runtime, approximation errors, and specific requirements of metrics, such as non-zero gradients. In this work, we introduce contextual decomposition for transformers (CD-T) to build interpretable circuits in large language models. CD-T can produce circuits of arbitrary level of abstraction, and is the first able to produce circuits as fine-grained as attention heads at specific sequence positions efficiently. CD-T consists of a set of mathematical equations to isolate contribution of model features. Through recursively computing contribution of all nodes in a computational graph of a model using CD-T followed by pruning, we are able to reduce circuit discovery runtime from hours to seconds compared to state-of-the-art baselines. On three standard circuit evaluation datasets (indirect object identification, greater-than comparisons, and docstring completion), we demonstrate that CD-T outperforms ACDC and EAP by better recovering the manual circuits with an average of 97% ROC AUC under low runtimes. In addition, we provide evidence that faithfulness of CD-T circuits is not due to random chance by showing our circuits are 80% more faithful than random circuits of up to 60% of the original model size. Finally, we show CD-T circuits are able to perfectly replicate original models' behavior (faithfulness $ = 1$) using fewer nodes than the baselines for all tasks. Our results underscore the great promise of CD-T for efficient automated mechanistic interpretability, paving the way for new insights into the workings of large language models.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
