连续思维的涌现：揭示连续思维链的训练动态
Emergence of Superposition: Unveiling the Training Dynamics of Chain of Continuous Thought
AI总结：
本文通过分析两层Transformer在有向图可达性问题上的训练动态，揭示了连续思维链中叠加机制如何在训练过程中通过自回归扩展思维和预测阶段自然产生。
摘要：
先前的研究表明，连续思维链（连续CoT）通过使隐式并行思维成为可能，提高了大型语言模型（LLMs）的推理能力，随后的一项研究通过展示一个配备连续CoT的两层Transformer可以通过维持多个推理轨迹的叠加来高效解决有向图可达性问题，提供了理论见解。然而，仍然不清楚如何通过基于梯度的训练方法自然地学习叠加机制。为填补这一空白，我们对简化后的两层Transformer在有向图可达性问题上的训练动态进行了理论分析，以揭示训练过程中叠加机制如何在两个训练阶段中出现——（i）一个生成思维的阶段，该阶段通过自回归的方式扩展连续思维，以及（ii）一个将思维转换为最终答案的预测阶段。我们的分析揭示，在使用连续思维进行训练时，在温和假设下，索引匹配logit，一个反映模型局部搜索能力强度的重要量值，会首先增加然后保持有界。有界的索引匹配logit在推理过程中有效地平衡了探索和利用：模型将利用局部问题结构来识别可能的搜索轨迹，并在不确定正确解决方案时，对多个此类轨迹分配可比的权重以进行探索，从而产生叠加。我们的实验结果跟踪logit的增长进一步验证了我们的理论。
英文摘要：
Previous work shows that the chain of continuous thought (continuous CoT) improves the reasoning capability of large language models (LLMs) by enabling implicit parallel thinking, and a subsequent work provided theoretical insight by showing that a two-layer transformer equipped with continuous CoT can efficiently solve directed graph reachability by maintaining a superposition of multiple reasoning traces in the continuous thought. However, it remains unclear how the superposition mechanism is naturally learned from gradient-based training methods. To fill this gap, we theoretically analyze the training dynamics of a simplified two-layer transformer on the directed graph reachability problem to unveil how the superposition mechanism emerges during training in two training stages -- (i) a thought-generation stage that autoregressively expands the continuous thought, and (ii) a prediction stage that converts the thought into the final answer. Our analysis reveals that during training using continuous thought, the index-matching logit, an important quantity which reflects the strength of the model's local search ability, will first increase and then remain bounded under mild assumptions. The bounded index-matching logit effectively balances exploration and exploitation during the reasoning process: the model will exploit local problem structures to identify plausible search traces, and assign comparable weights to multiple such traces to explore when it is uncertain about which solution is correct, which results in superposition. Our experimental results tracking the growth of logits further validate our theory.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
