从增长到循环:对大语言模型中迭代计算的统一视角
From Growing to Looping: A Unified View of Iterative Computation in LLMs
AI总结:
本文提出深度增长和循环是互补的实用方法,通过统一视角揭示大语言模型中迭代计算的机制,提升推理能力。
摘要:
循环,即在深度中重复使用一块层,以及深度增长,通过复制中间层来训练浅层到深层模型,都已被联系到更强的推理能力,但它们之间的关系仍不明确。我们提供了一种机制性的统一:循环和深度增长模型表现出收敛的深度特征,包括对晚期层的依赖增加以及与循环或增长块对齐的重复模式。这些共享的特征支持了它们的收益源于共同形式的迭代计算的观点。基于这一联系,我们展示了这两种技术是适应性和可组合的:在深度增长模型的中间块上应用推理时的循环可以提高某些推理原语的准确性,最高可达2倍,尽管模型从未被训练过循环。两种方法在获得更多上下文示例或额外监督微调数据时也适应得更好。此外,深度增长模型在使用高质量、数学密集型的冷却混合物时获得最大的推理收益,这可以通过适应中间块来循环进一步提升。总体而言,我们的结果将深度增长和循环定位为互补、实用的方法,用于诱导和扩展迭代计算以提高推理能力。
英文摘要:
Looping, reusing a block of layers across depth, and depth growing, training shallow-to-deep models by duplicating middle layers, have both been linked to stronger reasoning, but their relationship remains unclear. We provide a mechanistic unification: looped and depth-grown models exhibit convergent depth-wise signatures, including increased reliance on late layers and recurring patterns aligned with the looped or grown block. These shared signatures support the view that their gains stem from a common form of iterative computation. Building on this connection, we show that the two techniques are adaptable and composable: applying inference-time looping to the middle blocks of a depth-grown model improves accuracy on some reasoning primitives by up to $2\times$, despite the model never being trained to loop. Both approaches also adapt better than the baseline when given more in-context examples or additional supervised fine-tuning data. Additionally, depth-grown models achieve the largest reasoning gains when using higher-quality, math-heavy cooldown mixtures, which can be further boosted by adapting a middle block to loop. Overall, our results position depth growth and looping as complementary, practical methods for inducing and scaling iterative computation to improve reasoning.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。