在潜空间中思考：用于大语言模型隐式推理的自适应锚点细化
Thinking in Latents: Adaptive Anchor Refinement for Implicit Reasoning in LLMs
AI总结：
本文提出AdaAnchor框架，通过自适应终止机制优化潜空间推理，提升准确率并减少计算步骤，实验表明在数学问题基准上准确率提升5%且减少48-60%的潜空间步骤。
摘要：
Token级链式推理（CoT）提示已成为激发大语言模型（LLMs）多步推理的标准方法，尤其适用于数学应用问题。然而，生成长中间轨迹会增加输出长度和推理成本，当模型能通过少量说明得到正确答案时，这种方法可能效率低下。为此，本文引入了潜空间推理方法，将计算转移到隐藏表示中，并仅输出最终答案。然而，许多潜空间推理方法依赖于固定的潜空间细化步骤数，这增加了另一个超参数，需要在不同模型和数据集上进行调优以平衡准确性和效率。本文提出AdaAnchor，一种潜空间推理框架，通过细化附加在输入上的潜空间锚点向量进行静默迭代计算。AdaAnchor进一步结合了自适应终止机制，监控锚点稳定性并终止细化过程，当锚点动态收敛时停止细化。在相同最大步骤预算下，AdaAnchor为简单实例分配更少步骤，为更难实例保留额外细化步骤。实验表明，在三个数学应用问题基准上，AdaAnchor在固定步骤潜空间细化上准确率提升高达5%，同时减少48-60%的平均潜空间步骤。与标准推理基线相比，AdaAnchor通过将计算转移到静默潜空间细化中，生成的token减少92-93%，提供了不同的准确率-效率权衡，同时显著降低输出token的使用量。
英文摘要：
Token-level Chain-of-Thought (CoT) prompting has become a standard way to elicit multi-step reasoning in large language models (LLMs), especially for mathematical word problems. However, generating long intermediate traces increases output length and inference cost, and can be inefficient when the model could arrive at the correct answer without extensive verbalization. This has motivated latent-space reasoning approaches that shift computation into hidden representations and only emit a final answer. Yet, many latent reasoning methods depend on a fixed number of latent refinement steps at inference, adding another hyperparameter that must be tuned across models and datasets to balance accuracy and efficiency. We introduce AdaAnchor, a latent reasoning framework that performs silent iterative computation by refining a set of latent anchor vectors attached to the input. AdaAnchor further incorporates an adaptive halting mechanism that monitors anchor stability across iterations and terminates refinement once the anchor dynamics converge, allocating fewer steps to easier instances while reserving additional refinement steps for harder ones under a shared maximum-step budget. Our empirical evaluation across three mathematical word-problem benchmarks shows that AdaAnchor with adaptive halting yields accuracy gains of up to 5% over fixed-step latent refinement while reducing average latent refinement steps by 48-60% under the same maximum-step budget. Compared to standard reasoning baselines, AdaAnchor achieves large reductions in generated tokens (92-93%) by moving computation into silent latent refinement, offering a different accuracy-efficiency trade-off with substantially lower output-token usage.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
