多智能体大语言模型对话中的决策协议
Decision Protocols in Multi-Agent Large Language Model Conversations
AI总结：
研究多智能体大语言模型对话中的决策协议，通过MALLM框架实现并评估投票、共识和判断决策机制，考察其对多种任务的影响，发现共识协议在知识密集型领域出色，投票和判断协议对逻辑任务有效，增加响应多样性可提高决策质量。
摘要：
提高大语言模型（LLMs）的任务性能至关重要，但扩展这些模型面临收益递减和成本高等重大挑战。多智能体系统（MAS）通过在专门智能体之间分配任务来提高整体任务性能，提供了一个有前景的解决方案。决策协议是MAS的关键组件，它指定多个智能体如何协作以创建最终解决方案。本文介绍了多智能体LLM（MALLM）框架，该框架实现并评估了各种决策协议，即投票、共识和判断决策机制，以模拟用于对话任务解决的多智能体讨论。与以往工作不同，本研究系统地考察了它们对从基于知识的数据集（MMLU、MMLU - Pro、GPQA）和基于逻辑的数据集（StrategyQA、MuSR、Math - lvl - 5、SQuAD 2.0）等各种任务的影响。结果表明，共识协议在知识密集型领域表现出色，而投票和判断协议对基于逻辑的任务更有效。通过独立生成解决方案增加响应多样性可提高决策质量，而决策过程中信息访问的变化影响最小。
英文摘要：
Improving the task performance of Large Language Models (LLMs) is essential, yet scaling these models faces significant challenges such as diminishing returns and high costs. Multi-Agent Systems (MAS) offer a promising solution by distributing tasks among specialized agents to improve the overall task performance. This can reduce training costs at the expense of increased test time due to the discussion and decision-making process. The decision protocol is a critical component of MAS because it specifies how multiple agents collaborate to create a final solution. This thesis introduces the Multi-Agent LLM (MALLM) framework, which implements and evaluates various decision protocols, namely voting, consensus, and judge decision mechanisms, to simulate multi-agent discussions for conversational task solving. Unlike previous work that used a single decision protocol or tested them on limited datasets, this study systematically examines their impact on a diverse set of tasks, ranging from knowledge-based datasets (MMLU, MMLU-Pro, GPQA) and logic-based datasets (StrategyQA, MuSR, Math-lvl-5, SQuAD 2.0). The results indicate that consensus protocols excel in knowledge-intensive domains while voting and judge protocols are more effective for logic-based tasks. Increasing response diversity through independent solution generation improves decision quality, while changes in information access during the decision process have minimal impact.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
