arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2405.15071cs.CL
收藏

经过Grokking的Transformer是隐式推理器:通向泛化边界的机制之旅

Grokked Transformers are Implicit Reasoners: A Mechanistic Journey to the Edge of Generalization

The Ohio State University(俄亥俄州立大学);Carnegie Mellon University(卡内基梅隆大学)

Boshi Wang, Xiang Yue, Yu Su, Huan Sun

AI总结:

该研究发现Transformer可通过grokking学会隐式推理,不同推理类型泛化能力有差异,揭示了其内部机制,为架构改进提供方向,且完全grokked的Transformer在复杂推理上优于主流大模型。

摘要:

我们研究Transformer是否能够学会基于参数化知识进行隐式推理——这是即便最强大的语言模型也难以掌握的技能。聚焦组合推理与比较推理这两类代表性推理类型,我们一致发现Transformer能够学会隐式推理,但只能通过grokking(即远超过拟合阶段的长时间训练)实现。不同推理类型的泛化水平也存在差异:面对分布外样本时,Transformer在组合推理上无法实现系统性泛化,但在比较推理上能够成功。我们在整个训练过程中深入探究模型内部机制,通过分析实验揭示:1)grokking背后的机制,例如泛化回路的形成,以及其与泛化回路和记忆回路的相对效率的关系;2)系统性与泛化回路配置之间的关联。我们的发现为更好地诱导隐式推理的数据与训练设置提供了指导,并提出了Transformer架构的潜在改进方向,例如鼓励跨层知识共享。此外,我们证明,对于一个搜索空间巨大的高难度推理任务,基于非参数记忆的GPT-4-Turbo和Gemini-1.5-Pro无论采用何种提示风格或检索增强手段都表现极差,而完全经过grokking的Transformer能够达到接近完美的准确率,展现了参数化记忆在复杂推理中的强大能力。

英文摘要:

We study whether transformers can learn to implicitly reason over parametric knowledge, a skill that even the most capable language models struggle with. Focusing on two representative reasoning types, composition and comparison, we consistently find that transformers can learn implicit reasoning, but only through grokking, i.e., extended training far beyond overfitting. The levels of generalization also vary across reasoning types: when faced with out-of-distribution examples, transformers fail to systematically generalize for composition but succeed for comparison. We delve into the model's internals throughout training, conducting analytical experiments that reveal: 1) the mechanism behind grokking, such as the formation of the generalizing circuit and its relation to the relative efficiency of generalizing and memorizing circuits, and 2) the connection between systematicity and the configuration of the generalizing circuit. Our findings guide data and training setup to better induce implicit reasoning and suggest potential improvements to the transformer architecture, such as encouraging cross-layer knowledge sharing. Furthermore, we demonstrate that for a challenging reasoning task with a large search space, GPT-4-Turbo and Gemini-1.5-Pro based on non-parametric memory fail badly regardless of prompting styles or retrieval augmentation, while a fully grokked transformer can achieve near-perfect accuracy, showcasing the power of parametric memory for complex reasoning.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑