arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2410.20007cs.AIcs.CL
收藏

协作式战略规划增强大型语言模型的推理能力

Cooperative Strategic Planning Enhances Reasoning Capabilities in Large Language Models

Carnegie Mellon University(卡内基梅隆大学);Tsinghua University(清华大学)

Danqing Wang, Zhuorui Ye, Fei Fang, Lei Li

AI总结:

针对多智能体LLM协作不足的问题,本文提出CoPlanner框架,由规划智能体提供战略提示、推理智能体执行推断,并用PPO训练规划策略,显著提升多步骤推理性能。

摘要:

增强大型语言模型(LLM)的推理能力,对于使其能够处理复杂的多步骤问题至关重要。多智能体框架在提升LLM推理能力方面已展现出巨大潜力。然而,LLM智能体之间缺乏有效协作,阻碍了其性能表现,尤其是在多步骤推理任务中。本文提出了一种新颖的协作式多智能体推理框架(CoPlanner),通过分离推理步骤并为不同智能体分配不同职责来实现。CoPlanner由两个LLM智能体组成:规划智能体和推理智能体。规划智能体提供高层级战略提示,而推理智能体遵循这些提示并推断答案。通过在交互式推理过程中利用近端策略优化(PPO)训练规划智能体的策略,基于LLaMA-3-8B的CoPlanner在LogiQA上比此前最佳方法高出9.94%,在BBH上高出3.09%。我们的结果表明,规划智能体的指导以及智能体之间的有效协作,促成了CoPlanner在处理多步骤推理问题时的优越性能。

英文摘要:

Enhancing the reasoning capabilities of large language models (LLMs) is crucial for enabling them to tackle complex, multi-step problems. Multi-agent frameworks have shown great potential in enhancing LLMs' reasoning capabilities. However, the lack of effective cooperation between LLM agents hinders their performance, especially for multi-step reasoning tasks. This paper proposes a novel cooperative multi-agent reasoning framework (CoPlanner) by separating reasoning steps and assigning distinct duties to different agents. CoPlanner consists of two LLM agents: a planning agent and a reasoning agent. The planning agent provides high-level strategic hints, while the reasoning agent follows these hints and infers answers. By training the planning agent's policy through the interactive reasoning process via Proximal Policy Optimization (PPO), the LLaMA-3-8B-based CoPlanner outperforms the previous best method by 9.94\% on LogiQA and 3.09\% on BBH. Our results demonstrate that the guidance from the planning agent and the effective cooperation between the agents contribute to the superior performance of CoPlanner in tackling multi-step reasoning problems.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑