We test the robustness of debate as a method of scalable oversight by training models to debate with data generated via self-play. In a long-context reading comprehension task, we find that language model based evaluators answer questions more accurately when judging models optimized to win debates. By contrast, we find no such relationship for consultancy models trained to persuade a judge without an opposing debater present. In quantitative and qualitative comparisons between our debate models and novel consultancy baselines, we find evidence that debate training encourages stronger and more informative arguments, showing promise that it can help provide high-quality supervision for tasks that are difficult to directly evaluate.
Debatable Intelligence: Benchmarking LLM Judges via Debate Speech Evaluation
School of Computer Science and Engineering, The Hebrew University of Jerusalem（计算机科学与工程学院，特拉维夫大学）；IBM Research（IBM研究院）；The Allen Institute for AI (AI2)（人工智能研究所）
AI Debaters are More Persuasive when Arguing in Alignment with Their Own Beliefs
FAIR, IALAB, Universidad de Buenos Aires（FAIR、IALAB、布宜诺斯艾利斯大学）；Universidad de Buenos Aires（布宜诺斯艾利斯大学）；Universidad Nacional de Córdoba（科尔多瓦国立大学）；BAISH, Universidad de Buenos Aires（BAISH、布宜诺斯艾利斯大学）；Instituto de Investigación en Informática LIDI, Universidad Nacional de La Plata（信息研究所LIDI、拉普拉塔国立大学；CONICET）；CONICET（计算机科学与工程系，南大学及ICIC UNS-CONICET）；Dept. of Comp. Sci. and Eng., Universidad Nacional del Sur & ICIC UNS-CONICET（人工智能研究所（IIIA-CSIC），西班牙）；Artificial Intelligence Research Institute (IIIA-CSIC), ES
University of California, Los Angeles（加州大学洛杉矶分校）；Seoul National University（首尔国立大学）；Qatar Computing Research Institute（卡塔尔计算研究所）；Google（谷歌）；George Washington University（乔治华盛顿大学）；Stanford University（斯坦福大学）；Scale AI；University of Washington（华盛顿大学）
When to Trust Context: Self-Reflective Debates for Context Reliability
Brown University（布朗大学）；Stanford University（斯坦福大学）；University of New South Wales（新南威尔士大学）；Xi’an University of Electronic Science and Technology（西安电子科技大学）；University of Chicago（芝加哥大学）
Self-signals Driven Multi-LLM Debate for Efficient and Accurate Reasoning
University of Cambridge（剑桥大学）；Sorbonne Université（索邦大学）；University of Science and Technology of China（中国科学技术大学）；Beihang University（北航大学）；Nanyang Technological University（南洋理工大学）
Does Multi-Agent Debate Improve AI Feedback on Research Papers?
Charles University, Faculty of Social Sciences, Institute of Economic Studies（查理大学社会科学学院经济研究所）；Charles University, Prague Centre for Economic Policy Research, London（查理大学布拉格经济政策研究中心，伦敦）；Meta-Research Innovation Center at Stanford（斯坦福大学元研究创新中心）
AgentsCourt: Building Judicial Decision-Making Agents with Court Debate Simulation and Legal Knowledge Augmentation
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）
Fair Fact-Checking: Closing the Cross-Lingual Gap in LLM Factual Judgement with RoSh
Shifa Tameer-e-Millat University（希法塔米尔米拉特大学）；BRAINS, Brandenburg Research Center for Applied Intelligent Systems（勃兰登堡应用智能系统研究中心）；Max Planck Institute for Security and Privacy（马克斯·普朗克安全与隐私研究所）；University of Cambridge（剑桥大学）；The Pennsylvania State University（宾夕法尼亚州立大学）；GISMA University of Applied Sciences（GISMA应用科学大学）
When in Doubt, Consult: Expert Debate for Sexism Detection via Confidence-Based Routing
Department of Computer Science, University of Warwick（沃里克大学计算机科学系）；College of Business Studies, Public Authority of Applied Education and Training, Kuwait（卡塔尔应用教育与培训公共机构商学院）
Counterfactual Debating with Preset Stances for Hallucination Elimination of LLMs
University of Science and Technology of China（中国科学技术大学）；National University of Singapore（新加坡国立大学）；Electronic Science Research Institute of China Electronics（中国电子科技集团电子科学研究院）
