发表机构
University of Technology Sydney; East China Normal University; Shanghai University of Electric Power; University of Agder; Squirrel AI Learning（悉尼科技大学; 华东师范大学; 上海电力大学; 阿格德大学; 松鼠AI）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本研究通过几何分析揭示循环Transformer中局部改进方向因步长过大导致有限步失败，并提出路径曲率分解与局部二次模型预测有效步长，实验表明固定四分之一步长在多数失败案例中带来正增益。
AI 中文摘要
循环Transformer通过复用共享层进行迭代潜在推理，提供了一种参数高效的测试时扩展途径。然而，额外的迭代可能会降低对参考答案的支持度，这使得我们难以判断更新方向是局部无益，还是其完整位移过大。我们通过沿模型自身的更新方向分析参考效用（衡量这种支持度的指标），并改变提供给读出层的提议位移比例，来研究这一区别。这揭示了有限步失败，即局部改进的方向会产生有害的完整更新。路径曲率分解刻画了初始进展如何丢失，而局部二次模型则预测了全步增益和有用的步长尺度。基于累积曲率变化的界刻画了这些预测的近似误差。在两个模型家族上的实验揭示了在数学和常识任务上的这种分离。在四个设置中，固定四分之一步长在72.2%至83.2%的选定失败案例中产生了参考效用的正增益。这些发现确定了更新方向与步长尺度之间的不匹配是进展丢失的一种机制，解释了为何一些有害更新保留了有用的计算。
英文摘要
Looped Transformers offer a parameter-efficient route to test-time scaling by reusing shared layers for iterative latent reasoning. However, additional iterations can reduce support for a reference answer, leaving unclear whether an update's direction is locally unhelpful or its full displacement moves too far. We study this distinction by analysing reference utility, which measures this support, along the model's own update direction, varying the fraction of the proposed displacement supplied to the readout. This reveals finite-step failures in which a locally improving direction produces a harmful full update. A pathwise curvature decomposition characterises how initial progress is lost, while a local quadratic model predicts full-step gains and useful step scales. Bounds based on accumulated curvature variation characterise the approximation error of these predictions. Experiments across two model families reveal this separation on mathematical and commonsense tasks. A fixed quarter step produces positive gains in reference utility for 72.2--83.2% of selected failures across four settings. These findings identify a mismatch between update direction and step scale as a mechanism of lost progress, explaining how some harmful updates retain useful computation.
