AJAR:适应性突破架构用于红队测试
AJAR: Adaptive Jailbreak Architecture for Red-teaming
AI总结:
AJAR框架通过可调用MCP服务实现多轮突破算法,提升X-Teaming攻击成功率至76.0%,并在工具访问下优化攻击面。
摘要:
大型语言模型(LLM)的安全评估正从内容审核转向行动安全,因为现代系统获得了持久状态、工具访问和自主控制回路。现有的突破框架仍然在适应多轮攻击和代理运行时之间存在差距:攻击算法通常被封装为单体脚本,而代理利用工具很少暴露显式的回滚、工具模拟或策略切换抽象。我们提出了AJAR,一个红队框架,它将多轮突破算法作为可调用的MCP服务暴露出来,并让审计代理在基于Petri构建的工具感知运行时中 orchestrate 它们。AJAR将三种代表性攻击,即Crescendo、ActorAttack和X-Teaming,集成到一个共享的服务接口下,用于规划、提示生成、优化、评估和上下文控制。在200个HarmBench验证行为上,AJAR将X-Teaming的攻击成功率(ASR)从65.0%提升到76.0%,在单轮中达到80%的累积成功率比原生实现早出现,且在PyRIT中更有效地重现Crescendo(91.0% vs. 87.5% ASR)。行为层面的分析显示,这些收益集中在困难类别中,且经常依赖于回滚启用的转录修复。我们进一步表明,工具访问重塑而不是均匀扩大攻击面:ActorAttack在有工具时ASR从51.0%提升到56.0%,而Crescendo从91.0%降至78.0%,X-Teaming从76.0%降至55.5%,在依赖长期语义构建的类别中下降最为明显。这些结果将AJAR定位为在现实代理约束下评估多轮突破的实用基础。代码和数据可在https://github.com/douyipu/ajar获取。
英文摘要:
Large language model (LLM) safety evaluation is moving from content moderation to action security as modern systems gain persistent state, tool access, and autonomous control loops. Existing jailbreak frameworks still leave a gap between adaptive multi-turn attacks and agentic runtimes: attack algorithms are usually packaged as monolithic scripts, while agent harnesses rarely expose explicit abstractions for rollback, tool simulation, or strategy switching. We present AJAR, a red-teaming framework that exposes multi-turn jailbreak algorithms as callable MCP services and lets an Auditor Agent orchestrate them inside a tool-aware runtime built on Petri. AJAR integrates three representative attacks, namely Crescendo, ActorAttack, and X-Teaming, under a shared service interface for planning, prompt generation, optimization, evaluation, and context control. On 200 HarmBench validation behaviors, AJAR improves X-Teaming from 65.0% to 76.0% attack success rate (ASR), reaches 80% cumulative success one turn earlier than the native implementation, and reproduces Crescendo more effectively than PyRIT (91.0% vs. 87.5% ASR). Behavior-level analysis shows that these gains are concentrated in hard categories and frequently depend on rollback-enabled transcript repair. We further show that tool access reshapes rather than uniformly enlarges the attack surface: ActorAttack rises from 51.0% to 56.0% ASR with tools, whereas Crescendo drops from 91.0% to 78.0% and X-Teaming from 76.0% to 55.5%, with the sharpest declines appearing in categories that rely on long semantic buildup. These results position AJAR as a practical foundation for evaluating multi-turn jailbreaks under realistic agent constraints. Code and data are available at https://github.com/douyipu/ajar.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。