arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2605.09504cs.CRcs.AIcs.LG
收藏

位置:AI安全政策应针对系统,而非模型

Restricting the Model, Missing the System: Measurement and Accountability in Offensive AI Governance

AI Safety Department(人工智能安全部门);SimulaMet and OsloMet(SimulaMet和奥斯陆计量)

Michael Alexander Riegler, Finn Schwall, Annika Willoch Olstad, Birk Sebastian Frostelid Torpmann-Hagen, Sushant Gautam, Klas H. Pettersen, Inga Strümke

AI总结:

本文提出swarm-attack框架,展示通过协调轻量级LLM代理发现系统漏洞和绕过安全机制的可能性,证明在低成本硬件上可实现零成本安全测试。

摘要:

我们提出了swarm-attack,一个开源对抗测试框架,其中多个轻量级LLM代理通过共享内存、并行探索和进化优化协调。我们的结果表明,通过使用商用硬件和公开模型,可以有效地绕过前沿模型的安全措施并发现软件漏洞,即推动Anthropic Mythos Preview受限发布的功能类别。我们报告了两个实验。在第一个实验中,五个120亿参数模型各自对GPT-4o和Claude Sonnet~4执行了225次 jailbreak 攻击。针对GPT-4o,swarm实现了45.8%的有效破坏率,产生49次严重破坏;尽管技术成功率为40%,但针对Claude Sonnet-4,有效破坏率为0%。在第二个实验中,相同模型对具有9个植入CWE的易受攻击C应用程序进行了联合源代码分析和二进制模糊测试。通过手工制作的漏洞种子集、正则表达式模式检测和基于AddressSanitizer的崩溃分类,该流程在大约四分钟内恢复了9个漏洞(100%召回率)在消费者MacBook上。当移除这些scaffold组件时,相同模型通过崩溃验证恢复0个漏洞,通过引用恢复2个漏洞。因此,推动Anthropic Mythos Preview受限发布的功能类别可以在几乎零成本下重现;重要的是系统scaffold本身,它补偿了小规模个体模型的有限推理能力。

英文摘要:

We show that the instruments used to measure AI offensive capability fail in two ways: (i) they overstate harm, and (ii) they credit the model with capability that belongs to the surrounding system. We argue that restricting access to a model is therefore necessary but not sufficient and that policy and procurement also need system-level, harm-grounded capability assessment. In June 2026, two frontier models were suspended under US export controls, reportedly prompted by a jailbreak that asked a model to read a codebase and fix its flaws. This finding measured an elicitation \emph{system} of model, prompt, and task. We support our argument with a study of an open-source framework in which lightweight large language model (LLM) agents coordinate through shared memory and evolutionary optimization, providing two pieces of evidence. First, jailbreak metrics overstate harm: over 225 swarm-generated attacks per target, LLM-as-judge scoring rated Claude Sonnet 4 compromised in 40% of attacks, yet manual verification found actionable harmful content in none, against a 45.8\% Effective Harm Rate for GPT-4o. Second, scaffolded evaluations misattribute capability: on a planted-vulnerability target, a full pipeline built around a 1.2B-parameter model recovers 9 of 9 weaknesses, while the same model without the hand-crafted components recovers 0 of 9 by crash verification and 2 of 9 by cited source line. Offensive capability is a property of model, scaffold, and evaluation protocol together. The duty to assess it should lie with whoever controls the system, shapes its behaviour, and can foresee what it will do. In our setting, that is the party who builds the harness around the model, a role that current regulation does not clearly cover.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑