无需言语的思考：通过抽象链式推理实现高效潜在推理
Thinking Without Words: Efficient Latent Reasoning with Abstract Chain-of-Thought
AI总结：
本文提出Abstract Chain-of-Thought，通过离散潜在推理机制在训练后实现高效推理，减少推理token数量同时保持性能，发现抽象词汇的幂律分布。
摘要：
尽管长的显式链式推理（CoT）在复杂推理任务中表现良好，但在推理过程中生成成本高。非言语推理方法通过利用连续表示生成较短序列，但性能仍低于显式CoT。我们提出Abstract Chain-of-Thought，一种离散潜在推理后训练机制，语言模型在生成响应前从保留词汇生成短序列。为使先前未见的抽象token有用，我们引入一种类似策略迭代的预热循环，交替进行（i）通过掩码和监督微调从显式CoT瓶颈化，以及（ii）通过受约束解码训练模型仅凭提示生成抽象token。预热后，我们通过受约束解码的warm-started强化学习优化抽象序列生成。Abstract-CoT在数学推理、指令遵循和多跳推理中表现出可比性能，同时减少11.6倍的推理token，并在不同语言模型家族中泛化。我们还发现抽象词汇的幂律分布，类似于自然语言中观察到的分布，随着训练阶段演变。我们的发现突显了后训练潜在推理机制在通过学习的抽象推理语言实现高效推理的潜力。
英文摘要：
While long, explicit chains-of-thought (CoT) have proven effective on complex reasoning tasks, they are costly to generate during inference. Non-verbal reasoning methods have emerged with shorter generation lengths by leveraging continuous representations, yet their performance lags behind verbalized CoT. We propose $\textbf{Abstract Chain-of-Thought}$, a discrete latent reasoning post-training mechanism in which the language model produces a short sequence of tokens from a reserved vocabulary in lieu of a natural language CoT, before generating a response. To make previously unseen ''abstract'' tokens useful, we introduce a policy iteration-style warm-up loop that alternates between (i.) bottlenecking from a verbal CoT via masking and performing supervised fine-tuning, and (ii.) self-distillation by training the model to generate abstract tokens from the prompt alone via constrained decoding with the codebook. After warm-up, we optimize the generation of abstract sequences with warm-started reinforcement learning under constrained decoding. Abstract-CoT achieves up to $11.6\times$ fewer reasoning tokens while demonstrating comparable performance across mathematical reasoning, instruction-following, and multi-hop reasoning, and generalizes across language model families. We also find an emergent power law distribution over the abstract vocabulary, akin to those seen in natural language, that evolves across the training phases. Our findings highlight the potential for post-training latent reasoning mechanisms that enable efficient inference through a learned abstract reasoning language.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
