arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2408.04472cs.CL
收藏

大型语言模型能在辩论中击败人类吗?一个用于竞争性辩论的动态多智能体框架

Can LLMs Beat Humans in Debating? A Dynamic Multi-agent Framework for Competitive Debate

Northeastern University(东北大学);Alibaba Group(阿里巴巴集团)

Yiqun Zhang, Xiaocui Yang, Shi Feng, Daling Wang, Yifei Zhang, Kaisong Song

AI总结:

本文提出动态多智能体框架Agent4Debate,通过四个协作智能体增强大型语言模型在竞争性辩论中的能力,实验显示其表现与人类相当。

摘要:

竞争性辩论是计算论证中一项复杂的任务。大型语言模型(LLMs)在该领域存在幻觉问题且缺乏竞争力。为应对这些挑战,我们提出了Agent for Debate(Agent4Debate),一个基于大型语言模型的动态多智能体框架,旨在增强其在竞争性辩论中的能力。受人类在辩论准备和执行中的行为启发,Agent4Debate采用了一种协作架构,其中四个专门智能体,即搜索者(Searcher)、分析者(Analyzer)、撰写者(Writer)和评审者(Reviewer),动态地交互与协作。这些智能体贯穿辩论全过程,覆盖从初始研究和论点构建到反驳与总结的多个阶段。为全面评估框架性能,我们构建了竞争性辩论竞技场(Competitive Debate Arena),包含66个精心挑选的中文辩题。我们招募了十位经验丰富的人类辩手,并收集了涉及Agent4Debate、基线模型和人类的200场辩论记录。评估采用Debatrix自动评分系统和专业人类评审员,基于已建立的Debatrix-Elo和Human-Elo排名。实验结果表明,最先进的Agent4Debate展现出与人类相当的能力。此外,消融研究证明了智能体结构中每个组件的有效性。

英文摘要:

Competitive debate is a complex task of computational argumentation. Large Language Models (LLMs) suffer from hallucinations and lack competitiveness in this field. To address these challenges, we introduce Agent for Debate (Agent4Debate), a dynamic multi-agent framework based on LLMs designed to enhance their capabilities in competitive debate. Drawing inspiration from human behavior in debate preparation and execution, Agent4Debate employs a collaborative architecture where four specialized agents, involving Searcher, Analyzer, Writer, and Reviewer, dynamically interact and cooperate. These agents work throughout the debate process, covering multiple stages from initial research and argument formulation to rebuttal and summary. To comprehensively evaluate framework performance, we construct the Competitive Debate Arena, comprising 66 carefully selected Chinese debate motions. We recruit ten experienced human debaters and collect records of 200 debates involving Agent4Debate, baseline models, and humans. The evaluation employs the Debatrix automatic scoring system and professional human reviewers based on the established Debatrix-Elo and Human-Elo ranking. Experimental results indicate that the state-of-the-art Agent4Debate exhibits capabilities comparable to those of humans. Furthermore, ablation studies demonstrate the effectiveness of each component in the agent structure.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑