思想多样性在多智能体辩论框架中增强了推理能力
Diversity of Thought Elicits Stronger Reasoning Capabilities in Multi-Agent Debate Frameworks
AI总结：
研究通过多智能体辩论框架发现，思想多样性显著提升LLM推理能力，中等规模模型在GSM-8K基准测试中超越GPT-4，达到91%准确率，同时在ASDiv基准测试中创下新纪录。
摘要：
大型语言模型（LLMs）在自然语言生成方面表现出色，但在数学推理任务中常自信产生错误回答。链式思维提示、自我验证和多智能体辩论是改进LLM推理和事实准确性的重要策略。基于Du等人的多智能体辩论框架，我们发现多智能体辩论在任何模型规模下均有效，且思想多样性显著增强辩论LLM的推理能力。在各种模型规模下，数学推理任务的表现最受益于使用多样化的训练模型。令人惊讶的是，经过四轮辩论后，一组中等能力的模型（Gemini-Pro、Mixtral 7BX8和PaLM 2-M）在GSM-8K基准测试中超越GPT-4，准确率达91%。相比之下，使用三个Gemini-Pro实例时，表现仅达到82%。最后，这组多样化的中等能力模型在ASDiv基准测试中创下新纪录（94%）。这些结果强调了AI的未来是代理的，多样化合作的代理能够产生超越最强大单个模型的涌现能力。
英文摘要：
Large language models (LLMs) excel in natural language generation but often confidently produce incorrect responses, especially in tasks like mathematical reasoning. Chain-of-thought prompting, self-verification, and multi-agent debate are among the strategies proposed to improve the reasoning and factual accuracy of LLMs. Building on Du et al.'s multi-agent debate framework, we find that multi-agent debate helps at any model scale, and that diversity of thought elicits stronger reasoning in debating LLMs. Across various model sizes, performance on mathematical reasoning tasks benefits most when diverse trained models are used. Remarkably, after 4 rounds of debate, a diverse set of medium-capacity models (Gemini-Pro, Mixtral 7BX8, and PaLM 2-M) outperforms GPT-4 on the GSM-8K benchmark, scoring 91% accuracy. By comparison, when 3 instances of Gemini-Pro are used, performance only reaches 82%. Finally, this diverse set of medium-capacity models sets a new state-of-the-art performance on the ASDiv benchmark (94%). These results underscore the idea that the future of AI is agentic, with diverse cooperating agents yielding emergent capabilities beyond even the most powerful individual models.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
