思想即规划：通过强化规划进行思维链优化的潜在世界模型
Thoughts-as-Planning: Latent World Models for Chain-of-Thoughts Optimization via Reinforcement Planning
AI总结：
提出Thoughts-as-Planning框架，将思维链优化形式化为潜在语义空间中的序贯决策过程，通过潜在世界模型模拟推理链编辑对下游输出的影响，并利用梯度下降或强化学习进行规划，在语言理解和生成任务上优于现有基线。
摘要：
大型语言模型（LLMs）在多种NLP任务上的成功提升了推理链优化作为对齐模型行为与任务目标的关键步骤的重要性。现有的推理链调优方法通常依赖于黑盒启发式或免梯度搜索，缺乏可解释性、泛化能力和样本效率。在这项工作中，我们引入了 extbf{思想即规划}，一个新颖的框架，将推理链优化形式化为潜在语义空间上的序贯决策过程。我们将LLM建模为部分可观测环境，并学习一个潜在世界模型来模拟推理链编辑对下游输出的影响。构建了一个保持邻近性的嵌入空间来编码推理链-响应动态，从而通过梯度下降或强化学习实现规划。我们的方法支持多尺度抽象，允许在token、片段和指令级别进行推理链编辑，并集成到统一规划器中。通过在语言理解和生成任务上的大量实验，我们证明了思想即规划在效率、鲁棒性和泛化性方面优于最先进的推理链调优基线，同时通过其结构化规划轨迹提供了可解释性。我们的代码可在https://github.com/FastLM/Thoughts-as-Planning获取。
英文摘要：
The success of large language models (LLMs) across diverse NLP tasks has elevated the importance of reasoning chain optimization as a critical step in aligning model behavior with task objectives. Existing reasoning chain tuning methods often rely on black-box heuristics or gradient-free search, which lack interpretability, generalization, and sample efficiency. In this work, we introduce \textbf{Thoughts-as-Planning}, a novel framework that formalizes reasoning chain optimization as a sequential decision-making process over a latent semantic space. We model the LLM as a partially observable environment and learn a latent world model that simulates the effect of reasoning chain edits on downstream outputs. A proximity-preserving embedding space is constructed to encode reasoning chain-response dynamics, enabling planning via gradient descent or reinforcement learning. Our method supports multi-scale abstraction, allowing reasoning chain edits at token, segment, and instruction levels to be integrated into a unified planner. Through extensive experiments on language understanding and generation tasks, we demonstrate that Thoughts-as-Planning outperforms state-of-the-art reasoning chain tuning baselines in efficiency, robustness, and generalization, while offering interpretability through its structured planning trajectory. Our code is available at https://github.com/FastLM/Thoughts-as-Planning.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
