AI 中文总结
本文针对循环Transformer迭代增多时性能下降的问题，提出循环原生残差连接InfiLoop，在7M参数规模下，其在Sudoku-Extreme、ARC-AGI-2等推理任务上优于现有递归架构，且在超2万次测试迭代后仍持续提升。
AI 中文摘要
本文提出，循环Transformer需要专属的残差连接，以防止迭代次数增加时性能下降。研究发现，增加循环迭代会降低推理准确率：有噪的状态更新会覆盖正确的中间推导，甚至撤销已完成的解决方案，导致后续迭代需从已退化的表示中恢复丢失的信息——一旦早期循环因循环中的长程传播出现错误，后续循环往往难以修正。本文提出InfiLoop，一种循环原生残差连接，可学习保留哪些过去的计算以及接受多少新更新，它结合基于内容的加权与学习到的时间衰减，以维护循环状态的运行摘要，精确的流式循环使持久聚合内存随循环次数增长保持恒定，这种自适应更新可抑制不可靠提议并保留有用的中间状态。在大量推理任务中，7M参数的InfiLoop模型优于现有递归架构，在Sudoku-Extreme上达到97.9%的精确准确率，在ARC-AGI-2上达到13.6%的pass@2，值得注意的是，在Sudoku-Extreme上，InfiLoop在测试时循环超过20000个有效步骤后仍持续提升，表明增加深度可直接转化为更强的推理能力，代码可在该https URL获取。
英文摘要
In this paper, we argue that looped Transformers need their own residual connections to prevent performance degradation as the number of iterations grows. We observe that increasing loop iterations can reduce reasoning accuracy: noisy state updates overwrite correct intermediate deductions and even undo completed solutions. This leaves subsequent iterations to recover lost information from an already degraded representation: once an error arises in an earlier loop, often as a result of long-range propagation through the recurrence, later loops find it difficult to correct. In this paper, we introduce InfiLoop, a loop-native residual connection that learns which past computations to retain and how much to accept from each new update. InfiLoop combines content-based weighting with learned temporal decay to maintain a running summary of recurrent states. An exact streaming recurrence keeps its persistent aggregation memory constant as the loop count grows. The resulting adaptive update suppresses unreliable proposals and preserves useful intermediate states. Across extensive reasoning tasks, a 7M-parameter InfiLoop model outperforms existing recursive architectures, reaching 97.9% exact accuracy on Sudoku-Extreme, and 13.6% pass@2 on ARC-AGI-2. Notably, on Sudoku-Extreme, InfiLoop continues to improve with test-time looping beyond 20,000 effective steps, showing that added depth translates directly into stronger reasoning. Our code is available at https://github.com/pixeli99/InfiLoop.
