我们是否应该走向MAD？多智能体辩论策略在大型语言模型中的应用审视
Should we be going MAD? A Look at Multi-Agent Debate Strategies for LLMs
AI总结：
本研究基准测试多智能体辩论策略，发现其默认性能不及自洽性等方法，但经超参数调优可超越，并提出调整智能体一致性以提升性能。
摘要：
大型语言模型（LLMs）的最新进展凸显了其在各领域回应查询的潜力。然而，确保生成式智能体提供准确可靠的答案仍是一个持续的挑战。在此背景下，多智能体辩论（MAD）已成为增强LLMs真实性的有前景策略。我们对一系列辩论和提示策略进行基准测试，以探索成本、时间和准确性之间的权衡。重要的是，我们发现当前形式的多智能体辩论系统并不能可靠地优于其他提出的提示策略，如自洽性和使用多条推理路径的集成方法。然而，在进行超参数调优时，多个MAD系统（如Multi-Persona）表现更好。这表明MAD协议可能并非天生劣于其他方法，而是对不同的超参数设置更为敏感且难以优化。我们基于这些结果提出改进辩论策略的见解，例如调整智能体一致性水平，这可以显著提升性能，甚至超越我们评估的所有其他非辩论协议。我们向社区提供一个开源仓库，其中包含多种最先进的协议及评估脚本，以便在流行的研究数据集上进行基准测试。
英文摘要：
Recent advancements in large language models (LLMs) underscore their potential for responding to inquiries in various domains. However, ensuring that generative agents provide accurate and reliable answers remains an ongoing challenge. In this context, multi-agent debate (MAD) has emerged as a promising strategy for enhancing the truthfulness of LLMs. We benchmark a range of debating and prompting strategies to explore the trade-offs between cost, time, and accuracy. Importantly, we find that multi-agent debating systems, in their current form, do not reliably outperform other proposed prompting strategies, such as self-consistency and ensembling using multiple reasoning paths. However, when performing hyperparameter tuning, several MAD systems, such as Multi-Persona, perform better. This suggests that MAD protocols might not be inherently worse than other approaches, but that they are more sensitive to different hyperparameter settings and difficult to optimize. We build on these results to offer insights into improving debating strategies, such as adjusting agent agreement levels, which can significantly enhance performance and even surpass all other non-debate protocols we evaluated. We provide an open-source repository to the community with several state-of-the-art protocols together with evaluation scripts to benchmark across popular research datasets.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
