arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2402.02619cs.LGcs.CL
收藏

理解变换器中的加法与减法

Understanding Addition and Subtraction in Transformers

University of Oxford(牛津大学);NTU(南洋理工大学);Martian(火星)

Philip Quirke, Clement Neo, Fazl Barez

AI总结:

本文通过训练和分析变换器模型,揭示了加法和减法的因果结构,并提出了新的精确算法。

摘要:

我们使用整数加法和减法作为受控且可精确求解的测试平台,以确定低损失变换器实现的算法可以确定地说什么——在逻辑和机械上。我们从头开始训练小型变换器(2-3层),发现它们在长进位和借位级联情况下的失败点,并将这些情况添加到训练数据中;大多数由此产生的模型在不到一小时的时间内达到>99.999%的准确率,解决如555555555+444444448=+1000000003之类的级联问题。对这些准确模型的可解释性分析——不是事先猜测——揭示了我们从中推导出的新型、严谨、精确的从左到右算法的结构,这些算法与变换器必须使用的因果顺序相匹配。然后我们将这些算法放回模型中:在46个模型中,这些模型在种子、深度、数字长度和操作上各不相同,准确的模型共享一个电路骨架,并在代表性的模型中通过更深入的因果分析(激活和边缘路径修补,带有控制)确认了多数字解决进位的子任务,该任务先前的分析留有疑问——同时排除了几个候选机制。骨架在模型大小和操作中反复出现,混合模型节点在加法和减法中无论是否从加法模型初始化还是从头训练,都是多义的。我们发布了这些模型、分析以及可重复的可解释性工具包。

英文摘要:

We use integer addition and subtraction as a controlled, exactly-solvable testbed for what can be said with confidence about the algorithm a low-loss transformer implements - logically and mechanically. We train small transformers (2-3 layers) from scratch, find the edge cases they fail (long carry and borrow cascades), and enrich the training data with them; most resulting models reach >$99.999% accuracy on 5-15 digit problems in under an hour, solving cascades such as 555555555+444444448=+1000000003. Interpretability analysis of these accurate models - not an a-priori guess - surfaces the structure from which we derive novel, sound, exact left-to-right algorithms for both operations, matching the causal ordering transformers must use. We then put these algorithms back to the models: across 46 models varying in seed, depth, digit-length, and operation, the accurate ones share one circuit skeleton, and in representative models deeper causal analysis (activation and edge path-patching, with controls) confirms the multi-digit resolved-carry sub-task that earlier analyses left open - while ruling out several candidate mechanisms. The skeleton recurs across model sizes and operations, and mixed-model nodes are polysemantic across addition and subtraction whether initialized from an addition model or trained from scratch. We release the models, analysis, and a reproducible interpretability toolkit.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑