arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2404.07129cs.LG
收藏

归纳头(induction head)的正常运行需要哪些条件?对上下文学习电路及其形成机制的机理研究

What needs to go right for an induction head? A mechanistic study of in-context learning circuits and their formation

Gatsby Computational Neuroscience Unit, University College London(盖茨比计算神经科学单元,伦敦大学学院);Google DeepMind(谷歌DeepMind)

Aaditya K. Singh, Ted Moskovitz, Felix Hill, Stephanie C. Y. Chan, Andrew M. Saxe

AI总结:

该研究针对Transformer中归纳头的多样性与涌现机制不明的问题,提出受光遗传学启发的因果框架,通过合成数据训练识别出三个驱动归纳头形成、引发损失相变的子电路,为理解上下文学习的机制提供了新视角。

摘要:

上下文学习是Transformer模型中一种强大的涌现能力。此前的机制可解释性研究已识别出一种可能对上下文学习至关重要的电路元件——归纳头(induction head, IH),它执行匹配-复制操作。在大型Transformer基于自然语言数据训练的过程中,IH的涌现时间与损失函数出现显著相变的时间大致重合。尽管已有关于IH的可靠证据,且其与相变的这一有趣关联值得关注,但学界对IH的多样性和涌现动力学的了解仍然相对有限:为何存在不止一个IH?它们之间的依赖关系如何?IH为何会突然出现?促成其涌现的子电路有哪些?我们通过在合成数据上训练、在受控环境中研究IH的涌现动力学,来解答这些问题。在此过程中,我们开发并分享了一种受光遗传学启发的新型因果框架,用于在整个训练过程中修改激活值。利用该框架,我们阐明了IH的多样性与加性特性。通过在训练全程钳制部分激活值子集,我们进一步识别出三个相互作用以驱动IH形成、进而引发相变的底层子电路。此外,这些子电路还揭示了IH形成过程中依赖数据的特性,例如相变的发生时间,这已然表明,深入理解这些需要“正常运作”才能促成归纳头形成的子电路具有重要前景。

英文摘要:

In-context learning is a powerful emergent ability in transformer models. Prior work in mechanistic interpretability has identified a circuit element that may be critical for in-context learning -- the induction head (IH), which performs a match-and-copy operation. During training of large transformers on natural language data, IHs emerge around the same time as a notable phase change in the loss. Despite the robust evidence for IHs and this interesting coincidence with the phase change, relatively little is known about the diversity and emergence dynamics of IHs. Why is there more than one IH, and how are they dependent on each other? Why do IHs appear all of a sudden, and what are the subcircuits that enable them to emerge? We answer these questions by studying IH emergence dynamics in a controlled setting by training on synthetic data. In doing so, we develop and share a novel optogenetics-inspired causal framework for modifying activations throughout training. Using this framework, we delineate the diverse and additive nature of IHs. By clamping subsets of activations throughout training, we then identify three underlying subcircuits that interact to drive IH formation, yielding the phase change. Furthermore, these subcircuits shed light on data-dependent properties of formation, such as phase change timing, already showing the promise of this more in-depth understanding of subcircuits that need to "go right" for an induction head.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑