竞争性辩论是计算论证中一项复杂的任务。大型语言模型(LLMs)在该领域存在幻觉问题且缺乏竞争力。为应对这些挑战,我们提出了Agent for Debate(Agent4Debate),一个基于大型语言模型的动态多智能体框架,旨在增强其在竞争性辩论中的能力。受人类在辩论准备和执行中的行为启发,Agent4Debate采用了一种协作架构,其中四个专门智能体,即搜索者(Searcher)、分析者(Analyzer)、撰写者(Writer)和评审者(Reviewer),动态地交互与协作。这些智能体贯穿辩论全过程,覆盖从初始研究和论点构建到反驳与总结的多个阶段。为全面评估框架性能,我们构建了竞争性辩论竞技场(Competitive Debate Arena),包含66个精心挑选的中文辩题。我们招募了十位经验丰富的人类辩手,并收集了涉及Agent4Debate、基线模型和人类的200场辩论记录。评估采用Debatrix自动评分系统和专业人类评审员,基于已建立的Debatrix-Elo和Human-Elo排名。实验结果表明,最先进的Agent4Debate展现出与人类相当的能力。此外,消融研究证明了智能体结构中每个组件的有效性。
英文摘要:
Competitive debate is a complex task of computational argumentation. Large Language Models (LLMs) suffer from hallucinations and lack competitiveness in this field. To address these challenges, we introduce Agent for Debate (Agent4Debate), a dynamic multi-agent framework based on LLMs designed to enhance their capabilities in competitive debate. Drawing inspiration from human behavior in debate preparation and execution, Agent4Debate employs a collaborative architecture where four specialized agents, involving Searcher, Analyzer, Writer, and Reviewer, dynamically interact and cooperate. These agents work throughout the debate process, covering multiple stages from initial research and argument formulation to rebuttal and summary. To comprehensively evaluate framework performance, we construct the Competitive Debate Arena, comprising 66 carefully selected Chinese debate motions. We recruit ten experienced human debaters and collect records of 200 debates involving Agent4Debate, baseline models, and humans. The evaluation employs the Debatrix automatic scoring system and professional human reviewers based on the established Debatrix-Elo and Human-Elo ranking. Experimental results indicate that the state-of-the-art Agent4Debate exhibits capabilities comparable to those of humans. Furthermore, ablation studies demonstrate the effectiveness of each component in the agent structure.
Debating Truth: Debate-driven Claim Verification with Multiple Large Language Model Agents
Department of Interactive Media, Hong Kong Baptist University(香港浸会大学互动媒体系);School of Computing and Data Science, The University of Hong Kong(香港大学计算与数据科学学院);College of Computer Science and Artificial Intelligence, Fudan University(复旦大学计算机科学与人工智能学院);Beijing Normal-Hong Kong Baptist University(北京师范大学-香港浸会大学联合国际学院);Shenzhen Institute of Advanced Technology(深圳先进技术研究院)
Counterfactual Debating with Preset Stances for Hallucination Elimination of LLMs
University of Science and Technology of China(中国科学技术大学);National University of Singapore(新加坡国立大学);Electronic Science Research Institute of China Electronics(中国电子科技集团电子科学研究院)
InspireDebate: Multi-Dimensional Subjective-Objective Evaluation-Guided Reasoning and Optimization for Debating
Key Laboratory of Embedded System and Service Computing, Ministry of Education, Tongji University(嵌入式系统与服务计算重点实验室,教育部,同济大学);School of Computer Science and Technology, Tongji University(计算机科学与技术学院,同济大学)
DS@GT at Touché: Large Language Models for Retrieval-Augmented Debate
Georgia Institute of Technology(佐治亚理工学院);Peoples' Friendship University of Russia (RUDN University)(俄罗斯人民友谊大学);Joint Institute for Nuclear Research(联合核研究所);Vrije Universiteit Amsterdam(阿姆斯特丹自由大学);University of Skövde(斯德哥尔摩大学)
Beyond Solo and Consistency: Vindicating Multi-Agent Debate via Conditional Progressive Pruning
Rutgers University, New Brunswick(罗格斯大学新布朗斯维克分校);University of Cambridge(剑桥大学);Tsinghua University(清华大学);Harvard University(哈佛大学);Case Western Reserve University(凯斯西储大学);University of California San Diego(加利福尼亚大学圣迭戈分校);Carnegie Mellon University(卡内基梅隆大学)
Debate on Graph: a Flexible and Reliable Reasoning Framework for Large Language Models
Xi’an Jiaotong University(西安交通大学);The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));Northwestern Polytechnical University(西北工业大学);Zhejiang Createlink Technology(浙江创联科技);Shandong University(山东大学)
DEBISS: a Corpus of Individual, Semi-structured and Spoken Debates
Federal University of Campina Grande (UFCG) - System(坎皮纳格雷联邦大学(UFCG)- 系统);Computing Campina Grande, Brazil Federal Institute of Paraíba (IFPB) Monteiro, Brazil(坎皮纳格雷计算,巴西帕拉州联邦理工学院(IFPB)蒙特伊罗,巴西)
LatticeMind: A Conflict-Aware Memory Primitive for Multi-Agent Systems
University of Science and Technology of China(中国科学技术大学);University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);Shanghai AI Laboratory(上海人工智能实验室);Beijing University of Posts and Telecommunications(北京邮电大学);University of Oxford(牛津大学)