几何潜在推理诱导LLM生成更短的文本
Geometric Latent Reasoning Induces Shorter Generations in LLMs
AI总结：
提出几何潜在推理（GLR）方法，通过轻量级过渡头在嵌入空间中预测迭代方向更新，近似离散推理轨迹，从而在不显式优化长度的情况下显著缩短LLM的生成步数。
摘要：
大型语言模型通过生成冗长的显式推理令牌链来解决复杂问题。虽然有效，但这使得推理成本高昂、对长度敏感，并受限于（离散的）自然语言。虽然潜在推理提供了连续的替代方案，但确定中间潜在状态的有用结构仍是一个开放挑战。在本文中，我们将潜在推理公式化为模型预训练令牌嵌入空间中的几何路径逼近问题。我们引入了几何潜在推理（GLR），它使用轻量级过渡头来预测嵌入空间中的迭代方向更新。利用文本思维链轨迹作为锚点，GLR学习逼近离散推理轨迹，同时允许偏离精确令牌嵌入的连续偏差。使用Qwen3模型在数学推理基准上的评估揭示了一个涌现现象：几何潜在推理在没有显式长度目标的情况下诱导出显著更短的生成。通过用连续的潜在步骤替换早期的显式推理，模型通常使用更少的总生成步骤达到正确答案。这些发现表明，连续轨迹充当紧凑的中间推理状态，揭示了潜在计算预算、输出长度和准确性之间的新权衡。
英文摘要：
Large language models solve complex problems by generating lengthy chains of explicit reasoning tokens. While effective, this makes reasoning expensive, length-sensitive, and constrained to (discrete) natural language. While latent reasoning offers a continuous alternative, determining useful structures for intermediate latent states is an open challenge. In this paper, we formulate latent reasoning as a geometric path-approximation problem within the model's pretrained token-embedding space. We introduce Geometric Latent Reasoning (GLR), which uses a lightweight transition head to predict iterative direction updates in embedding space. Using textual chain-of-thought traces as anchors, GLR learns to approximate discrete reasoning trajectories while permitting continuous deviations from exact token embeddings. Evaluations on mathematical reasoning benchmarks using Qwen3 models reveal an emergent phenomenon: geometric latent reasoning induces substantially shorter generations without an explicit length objective. By replacing early explicit reasoning with continuous latent steps, models often reach correct answers using substantially fewer total generation steps. These findings suggest that continuous trajectories act as compact intermediate reasoning states, exposing a new tradeoff between latent computation budget, output length, and accuracy.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
