潜在链式思考模型是否分步思考？对序列推理任务的机制研究
Do Latent-CoT Models Think Step-by-Step? A Mechanistic Study on Sequential Reasoning Tasks
AI总结：
本文研究了CODI模型在序列推理任务中的机制，揭示了潜在链式思考在不同跳数下的计算路径及可靠性问题。
摘要：
潜在链式思考（Latent-CoT）旨在实现无需生成长理性说明的分步计算，但其机制仍不清楚。我们研究了CODI，一个连续思考教师-学生蒸馏模型，在严格序列多项式迭代任务上的表现。使用logit-lens解码、线性探测、注意力分析和激活修补，我们定位了中间状态表示并追踪它们路由到最终读取的过程。在两跳和三跳任务中，CODI形成完整的桥梁状态集，这些状态能够在潜在思考位置上被解码，而最终输入则遵循一条单独的近直接路径；预测通过在思考结束边界处进行晚期融合产生。对于更长的跳数，CODI并不总是可靠地执行完整的潜在滚动，而是表现出一个专注于晚期中间步骤并将其与最后输入在答案读取位置融合的潜在推理路径。消融实验显示，这种路径在某些情况下会崩溃，包括更困难的优化。总体而言，我们阐明了CODI式潜在链式思考在何时能产生忠实的迭代计算，何时会采用压缩或捷径策略，并突显了设计稳健的潜在链式思考目标以应对序列推理的挑战。
英文摘要：
Latent Chain-of-Thought aims to enable step-by-step computation without emitting long rationales, yet its internal mechanisms remain unclear. We study CODI, a continuous-thought teacher-student distillation model, on strictly sequential polynomial-iteration tasks with known intermediate states. Using logit-lens decoding, linear probes, attention analysis, and activation patching, we localize intermediate-state representations and trace how they are routed to the final readout. In short-horizon, low-hop tasks, CODI forms faithful bridge states across latent-thought positions, while the final input follows a separate near-direct route; predictions arise through late fusion at the answer readout. As task depth and difficulty increase, however, CODI does not reliably sustain a full latent rollout: it either compresses computation into a partial late-intermediate pathway or, in harder regimes, loses the latent reasoning signature altogether. To explain this transition, we show theoretically that the task's algebraic structure controls its effective memory: compressible regimes support late-bottleneck reasoning, while incompressible regimes preserve full-history dependence and destabilize latent rollouts. Overall, our results characterize when CODI-style latent-CoT yields faithful iterative computation versus compressed or shortcut strategies.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
