arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2511.06396cs.AIcs.CR
收藏

通过多智能体辩论实现高效的LLM安全评估

Efficient LLM Safety Evaluation through Multi-Agent Debate

Beijing Institute of AI Safety and Governance(北京人工智能安全与治理研究院);Beijing Key Laboratory of Safe AI and Super Alignment(北京安全人工智能与超对齐重点实验室);Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);CSE, The Chinese University of Hong Kong(香港中文大学电子工程系);University of Chinese Academy of Sciences(中国科学院大学);Department of Mathematics, The Chinese University of Hong Kong(香港中文大学数学系);Long-term AI(长期人工智能)

Dachuan Lin, Guobin Shen, Zihao Yang, Tianrong Liu, Dongcheng Zhao, Yi Zeng

AI总结:

本文提出多智能体辩论框架,利用HAJailBench基准测试,提升LLM安全评估的可靠性与经济性,验证了少量辩论轮次即可获得显著效果。

摘要:

大型语言模型(LLM)的安全评估日益依赖LLM-as-judge流程,但强大的法官在大规模应用中仍昂贵。我们研究结构化的多智能体辩论是否能在保持基础模型大小和成本适度的情况下提高法官的可靠性。为此,我们引入了HAJailBench,一个包含11,100个标注交互的人工标注 Jailbreak 基准,覆盖了多种攻击方法和目标模型。我们将其与一个多智能体法官框架结合,其中批评者、防御者和法官在共享的安全标准下辩论。在HAJailBench上,该框架在匹配的小模型提示基线和先前的多智能体法官之上表现更优,同时在评估的定价快照下比GPT-4o更经济。消融结果进一步表明,少量辩论轮次足以捕获大部分收益。这些结果支持结构化、价值观一致的辩论作为可扩展LLM安全评估的实用设计。

英文摘要:

Safety evaluation of large language models (LLMs) increasingly relies on LLM-as-a-judge pipelines, but strong judges can still be expensive to use at scale. We study whether structured multi-agent debate can improve judge reliability while keeping backbone size and cost modest. To do so, we introduce HAJailBench, a human-annotated jailbreak benchmark with 11,100 labeled interactions spanning diverse attack methods and target models, and we pair it with a Multi-Agent Judge framework in which critic, defender, and judge agents debate under a shared safety rubric. On HAJailBench, the framework improves over matched small-model prompt baselines and prior multi-agent judges, while remaining more economical than GPT-4o under the evaluated pricing snapshot. Ablation results further show that a small number of debate rounds is sufficient to capture most of the gain. Together, these results support structured, value-aligned debate as a practical design for scalable LLM safety evaluation.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑