思维计划：基于启发式引导的大语言模型问题求解
Plan of Thoughts: Heuristic-Guided Problem Solving with Large Language Models
AI总结：
本文提出将多步推理建模为部分可观测马尔可夫决策过程，以语言模型对状态价值的反思作为启发式，并结合在线求解器POMCP，在24点游戏上实现89.4%的成功率，优于已有方法。
摘要：
尽管语言模型（LMs）在广泛领域的零样本推理任务中展现出显著能力，但在需要多步推理的问题上表现并不令人满意。此前缓解这一问题的方法是将较大的多步任务分解为子任务，并要求语言模型为每个子任务生成候选方案（“思维”），然后使用诸如深度优先搜索（DFS）之类的穷举规划方法来组合出解决方案。在本工作中，我们借鉴这一思想并引入两项新贡献：第一，我们形式化了一种基于规划的求解方法，通过部分可观测马尔可夫决策过程（POMDPs）利用语言模型执行多步问题求解，并将语言模型自身对状态价值的反思用作搜索启发式；第二，借助在线 POMDP 求解器 POMCP，我们在 24 点游戏（Game of 24）任务上取得了 89.4% 的成功率，优于现有方法，同时相比此前使用的固定树搜索展现出更好的随时性（anytime）性能特征。综合来看，这些贡献使现代语言模型能够更有效地分解并求解更大规模的推理任务。
英文摘要：
While language models (LMs) offer significant capability in zero-shot reasoning tasks across a wide range of domains, they do not perform satisfactorily in problems which requires multi-step reasoning. Previous approaches to mitigate this involves breaking a larger, multi-step task into sub-tasks and asking the language model to generate proposals ("thoughts") for each sub-task and using exhaustive planning approaches such as DFS to compose a solution. In this work, we leverage this idea to introduce two new contributions: first, we formalize a planning-based approach to perform multi-step problem solving with LMs via Partially Observable Markov Decision Processes (POMDPs), with the LM's own reflections about the value of a state used as a search heuristic; second, leveraging the online POMDP solver POMCP, we demonstrate a superior success rate of 89.4% on the Game of 24 task as compared to existing approaches while also offering better anytime performance characteristics than fixed tree-search which is used previously. Taken together, these contributions allow modern LMs to decompose and solve larger-scale reasoning tasks more effectively.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
