AI 中文总结
提出自回归思维流（ATF），将下一个连续思维建模为多模态分布，结合扩散头生成多样推理路径，在数学任务中提升准确性和覆盖率。
AI 中文摘要
推理问题通常允许多种有效的前进方式。连续推理有望将计算从语言标记扩展到更紧凑的潜在空间，但表示多种可能的下一步思考方式仍然困难。我们引入了自回归思维流（ATF），它将下一个连续思维建模为多模态分布。因果自回归模型执行推理计算，而轻量级扩散头从结果条件中生成一个合理的下一个思维。采样的思维被反馈到模型中，允许连续推理在可变步数内展开，同时保留预训练骨干。在数学推理任务中，ATF以紧凑的潜在轨迹提高了准确性，并从强化学习和额外的测试时思考中获益。多样本评估显示更广泛的解决方案覆盖，表明其多模态预测捕获了推理路径中的有用多样性。我们的结果表明，当多个可能的下一步思维保持可用而不是被压缩为单一预测时，连续推理更有效。
英文摘要
Reasoning problems often admit multiple valid ways to proceed. Continuous reasoning promises to move computation beyond language tokens into a more compact latent space, but representing several plausible ways to think next remains difficult. We introduce Autoregressive Thought Flow (ATF), which models the next continuous thought as a multimodal distribution. A causal autoregressive model performs the reasoning computation, while a lightweight diffusion head generates a plausible next thought from the resulting condition. The sampled thought is fed back into the model, allowing continuous reasoning to unfold for a variable number of steps while preserving the pretrained backbone. Across mathematical reasoning tasks, ATF improves accuracy with compact latent traces and benefits from reinforcement learning and additional test-time thinking. Multi-sample evaluation shows broader solution coverage, indicating that its multimodal predictions capture useful diversity among reasoning paths. Our results suggest that continuous reasoning is more effective when multiple possible next thoughts remain available rather than being collapsed into a single prediction.
Comments16 pages
