Efficient LLM Safety Evaluation through Multi-Agent Debate
Beijing Institute of AI Safety and Governance(北京人工智能安全与治理研究院);Beijing Key Laboratory of Safe AI and Super Alignment(北京安全人工智能与超对齐重点实验室);Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);CSE, The Chinese University of Hong Kong(香港中文大学电子工程系);University of Chinese Academy of Sciences(中国科学院大学);Department of Mathematics, The Chinese University of Hong Kong(香港中文大学数学系);Long-term AI(长期人工智能)
Safety evaluation of large language models (LLMs) increasingly relies on LLM-as-a-judge pipelines, but strong judges can still be expensive to use at scale. We study whether structured multi-agent debate can improve judge reliability while keeping backbone size and cost modest. To do so, we introduce HAJailBench, a human-annotated jailbreak benchmark with 11,100 labeled interactions spanning diverse attack methods and target models, and we pair it with a Multi-Agent Judge framework in which critic, defender, and judge agents debate under a shared safety rubric. On HAJailBench, the framework improves over matched small-model prompt baselines and prior multi-agent judges, while remaining more economical than GPT-4o under the evaluated pricing snapshot. Ablation results further show that a small number of debate rounds is sufficient to capture most of the gain. Together, these results support structured, value-aligned debate as a practical design for scalable LLM safety evaluation.
Aetheria: A multimodal interpretable content safety framework based on multi-agent debate and collaboration
Institute of Artificial Intelligence (TeleAI), China Telecom(人工智能研究院(TeleAI),中国电信);Sichuan University(四川大学);Peking University(北京大学);Beijing Jiaotong University(北京交通大学);Harbin Institute of Technology(哈尔滨工业大学);China Railway Rolling Stock Corporation Limited(中国铁路滚动股票有限公司)
MADRA: Multi-Agent Debate for Risk-Aware Embodied Planning
Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院);University of Chinese Academy of Sciences, Nanjing(中国科学院大学南京校区)
DynaDebate: Breaking Homogeneity in Multi-Agent Debate with Dynamic Path Generation
State Key Laboratory of Cognitive Intelligence, University of Science and Technology of China(认知智能国家重点实验室,中国科学技术大学);North Automatic Control Technology Institute(北自动控制技术研究所);Shenzhen Institute for Advanced Study, UESTC(深圳先进研究 institute, 电子科技大学)
The Truth Becomes Clearer Through Debate! Multi-Agent Systems with Large Language Models Unmask Fake News
Gaoling School of Artificial Intelligence, Renmin University of China(仁爱大学人工智能学院);Mohamed bin Zayed University of Artificial Intelligence(穆罕默德·本·扎耶德人工智能大学)
Where Reasoning Diverges: Localized Multi-Agent Debate for Multi-Hop Question Answering
The Chinese University of Hong Kong(香港中文大学);Nagoya University(名古屋大学);Institute of Science Tokyo(东京科学大学);University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校)
Jailbreaking Text-to-Image Models Through Cracks: Navigating Heterogeneous Safety Filters via Multi-Agent Debate
College of Computer Science and Technology, National University of Defense Technology(国防科技大学计算机科学与技术学院);Shenzhen MSU-BIT University(深圳北理莫斯科大学);City University of Hong Kong(香港城市大学)
Beyond Symmetric Agents: Cognitive Diversity and Multi-Agent Debate in Small Language Models
Boston Children’s Hospital, Harvard Medical School(波士顿儿童医院,哈佛医学院);Harvard School of Engineering and Applied Sciences, Harvard College(哈佛工程与应用科学学院,哈佛学院)
Efficient Multi-Agent Reasoning via Confidence-Guided Adaptive Debate
LG AI Research(LG AI研究院);School of Computer Science and Engineering, Chung-Ang University(中央大学计算机科学与工程学院);Division of Computer Engineering, Hankuk University of Foreign Studies(韩国外国语大学计算机工程系);Noah’s Farm(诺亚农场)
CortexDebate: Debating Sparsely and Equally for Multi-Agent Debate
School of Computer Science and Engineering, Nanjing University of Science and Technology(南京理工大学计算机科学与工程学院);College of Artificial Intelligence, Nanjing Agricultural University(南京农业大学人工智能学院);School of Automation and Intelligent Sensing, Shanghai Jiao Tong University(上海交通大学自动化与智能感知学院)
Breaking the Martingale Curse: Multi-Agent Debate via Asymmetric Cognitive Potential Energy
Mohamed bin Zayed University of Artificial Intelligence(莫扎德·本·扎耶德人工智能大学);Gaoling School of Artificial Intelligence, Renmin University of China(中国人民大学国际关系学院);Harvard University(哈佛大学)
Gradual Vigilance and Interval Communication: Enhancing Value Alignment in Multi-Agent Debates
National Engineering Research Center of Educational Big Data, Central China Normal University(华中师范大学教育大数据国家工程研究中心);Faculty of Artificial Intelligence in Education, Central China Normal University(华中师范大学人工智能教育学部)