LLMs 能提前多远规划？揭示思维链推理中的潜在视界
How Far Ahead Do LLMs Plan? Uncovering the Latent Horizon in Chain-of-Thought Reasoning
AI总结：
通过探测方法 Tele-Lens 研究 LLMs 在思维链推理中的潜在规划能力，发现其具有短视视界，并基于此提出利用稀疏枢轴位置增强不确定性估计及自动识别 CoT 绕过的假设。
摘要：
思维链推理已成为激发大型语言模型多步推理的核心机制。然而，近期证据呈现一种矛盾：隐藏状态似乎在 CoT 完全展开之前就已经编码了未来的推理，而显式步骤对于需要组合计算的任务仍然至关重要。为了加深对 LLM 内部状态与其言语化推理轨迹之间关系的理解，我们通过探测方法 Tele-Lens 研究了 LLMs 的潜在规划强度，该方法应用于跨不同任务领域的隐藏状态。我们的实证结果表明，LLMs 表现出短视视界，主要进行增量转换，而没有精确的全局规划。利用这一特性，我们提出了一个增强 CoT 不确定性估计的假设，并通过实验验证了一组稀疏的枢轴位置可以有效地代表整个路径的不确定性。我们进一步强调了利用 CoT 动态的重要性，并证明了可以在不降低性能的情况下实现 CoT 绕过的自动识别。我们的代码、数据和模型发布于 https://github.com/lxucs/tele-lens。
英文摘要：
Chain-of-thought (CoT) reasoning has become a central mechanism for eliciting multi-step reasoning in Large Language Models (LLMs). Yet recent evidence presents a tension: hidden states appear to already encode future reasoning before CoT fully unfolds, while explicit steps still remain crucial for tasks requiring compositional computation. To deepen the understanding between LLM's internal states and its verbalized reasoning trajectories, we investigate the latent planning strength of LLMs, through our probing method, Tele-Lens, applying to hidden states across diverse task domains. Our empirical results indicate that LLMs exhibit a myopic horizon, primarily conducting incremental transitions without precise global planning. Leveraging this characteristic, we propose a hypothesis on enhancing uncertainty estimation of CoT, which we validate that a sparse set of pivot positions can effectively represent the uncertainty of the entire path. We further underscore the significance of exploiting CoT dynamics, and demonstrate that automatic recognition of CoT bypass can be achieved without performance degradation. Our code, data and models are released at https://github.com/lxucs/tele-lens.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
