LLM鲁棒性排行榜v1——技术报告
LLM Robustness Leaderboard v1 --Technical report
AI总结：
本文介绍了PRISM Eval发布的LLM鲁棒性排行榜及自动化红队系统BET，该系统对37个顶尖模型实现100%攻击成功率，并提出细粒度鲁棒性指标与漏洞分析方法，揭示了模型攻击难度的巨大差异。
摘要：
本技术报告随PRISM Eval为Paris AI Action Summit发布的LLM鲁棒性排行榜一同发布。我们引入了PRISM Eval Behavior Elicitation Tool (BET)，这是一个通过Dynamic Adversarial Optimization执行自动化红队测试的AI系统，对41个最先进的LLM中的37个实现了100%的攻击成功率（ASR）。除了二元成功指标外，我们提出了一种细粒度鲁棒性指标，用于估计引发有害行为所需的平均尝试次数，揭示了尽管存在普遍漏洞，但各模型的攻击难度差异超过300倍。我们引入了原始级别漏洞分析，以识别哪些越狱技术对特定危害类别最有效。我们与来自AI Safety Network的受信任第三方进行的协作评估，展示了在社区中进行分布式鲁棒性评估的实用路径。
英文摘要：
This technical report accompanies the LLM robustness leaderboard published by PRISM Eval for the Paris AI Action Summit. We introduce PRISM Eval Behavior Elicitation Tool (BET), an AI system performing automated red-teaming through Dynamic Adversarial Optimization that achieves 100% Attack Success Rate (ASR) against 37 of 41 state-of-the-art LLMs. Beyond binary success metrics, we propose a fine-grained robustness metric estimating the average number of attempts required to elicit harmful behaviors, revealing that attack difficulty varies by over 300-fold across models despite universal vulnerability. We introduce primitive-level vulnerability analysis to identify which jailbreaking techniques are most effective for specific hazard categories. Our collaborative evaluation with trusted third parties from the AI Safety Network demonstrates practical pathways for distributed robustness assessment across the community.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
