投票还是共识?多智能体辩论中的决策机制
Voting or Consensus? Decision-Making in Multi-Agent Debate
AI总结:
本文系统比较多智能体辩论中的七种决策协议,发现投票更利于推理、共识更利于知识任务,并提出AAD与CI两种提升答案多样性的方法。
摘要:
多智能体辩论的成功在很大程度上取决于谨慎选择合适的参数。决策协议尤为突出,因为根据决策达成方式的不同,它会显著影响模型的最终答案。对决策协议进行系统比较十分困难,原因是许多研究除协议之外还改变了多个讨论参数。到目前为止,决策机制如何影响不同任务仍基本未知。本研究系统评估了七种决策协议(例如多数投票、一致共识)的影响。我们每次只改变一个变量——决策协议——以分析不同方法如何影响智能体之间的协作,并衡量知识任务与推理任务中的差异。结果表明,与其他决策协议相比,投票协议在推理任务中将性能提升13.2%,共识协议在知识任务中将性能提升2.8%。增加智能体数量可以提升性能,而投票前进行更多讨论轮次则会降低性能。为了通过提高答案多样性来改进决策,我们提出了两种新方法:All-Agents Drafting(AAD,全体智能体起草)和Collective Improvement(CI,集体改进)。我们的方法将任务性能提升最高达3.3%(AAD)和7.4%(CI)。本研究证明了在规模扩展之外,决策机制在多智能体辩论中的重要性。
英文摘要:
Much of the success of multi-agent debates depends on carefully choosing the right parameters. The decision-making protocol stands out as it can highly impact final model answers, depending on how decisions are reached. Systematic comparison of decision protocols is difficult because many studies alter multiple discussion parameters beyond the protocol. So far, it has been largely unknown how decision-making influences different tasks. This work systematically evaluates the impact of seven decision protocols (e.g., majority voting, unanimity consensus). We change only one variable at a time - the decision protocol - to analyze how different methods affect the collaboration between agents and measure differences in knowledge and reasoning tasks. Our results show that voting protocols improve performance by 13.2% in reasoning tasks and consensus protocols by 2.8% in knowledge tasks compared to other decision protocols. Increasing the number of agents improves performance, while more discussion rounds before voting reduce it. To improve decision-making by increasing answer diversity, we propose two new methods, All-Agents Drafting (AAD) and Collective Improvement (CI). Our methods improve task performance by up to 3.3% with AAD and up to 7.4% with CI. This work demonstrates the importance of decision-making in multi-agent debates beyond scaling.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。