OPTAGENT：通过言语强化学习优化多智能体大语言模型交互以增强推理能力
OPTAGENT: Optimizing Multi-Agent LLM Interactions Through Verbal Reinforcement Learning for Enhanced Reasoning
AI总结：
针对现有多智能体LLM推理系统交互质量被忽视、易压制弱势正确贡献的问题，提出OPTAGENT言语强化学习算法，通过动态优化协作结构与辩论通信质量，在多类推理任务上显著优于现有方法。
摘要：
大语言模型（LLM）在数学和科学任务中展现出了卓越的推理能力。为提升复杂推理能力，研究人员提出了多智能体系统以利用LLM智能体的集体智慧。然而，现有的协作结构要么是预定义的，要么依赖多数投票或圆桌辩论，这可能会压制正确但不那么占主导地位的智能体贡献。近期的方法将多智能体系统建模为图网络，但仅针对智能体性能进行优化，忽略了交互质量。我们假设有效的智能体通信对多智能体推理至关重要，且辩论质量发挥着重要作用。为解决这一问题，我们提出了$\text{OPTAGENT}$——一种多智能体言语强化学习算法，可动态构建并优化多智能体协作结构。我们的方法定义了动作空间和一种反馈机制，用于评估整个辩论过程中通信的稳健性和连贯性。最终决策通过所有智能体的多数投票得出。我们在数学推理、创意写作、科学推理和数值排序等多种推理任务上评估了$\text{OPTAGENT}$。结果表明，我们的方法在各类任务上显著优于单智能体提示方法和最先进的多智能体框架。
英文摘要：
Large Language Models (LLMs) have shown remarkable reasoning capabilities in mathematical and scientific tasks. To enhance complex reasoning, multi-agent systems have been proposed to harness the collective intelligence of LLM agents. However, existing collaboration structures are either predefined or rely on majority voting or round-table debates, which can suppress correct but less dominant agent contributions. Recent approaches model multi-agent systems as graph networks but optimize purely for agent performance, neglecting the quality of interactions. We hypothesize that effective agent communication is crucial for multi-agent reasoning and that debating quality plays a significant role. To address this, we propose $\ours$, a multi-agent verbal reinforcement learning algorithm that dynamically constructs and refines multi-agent collaboration structures. Our method defines action spaces and a feedback mechanism that evaluates communication robustness and coherence throughout the debate. The final decision is achieved through a majority vote over all the agents. We assess $\ours$ on various reasoning tasks, including mathematical reasoning, creative writing, scientific reasoning, and numerical sorting. Results demonstrate that our approach significantly outperforms single-agent prompting methods and state-of-the-art multi-agent frameworks on diverse tasks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
