arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2509.08729cs.CLcs.AI

X-Teaming 进化式 M2S:多轮到单轮越狱模板的自动发现

X-Teaming Evolutionary M2S: Automated Discovery of Multi-turn to Single-turn Jailbreak Templates

AIM Intelligence(AIM智能)

Hyunjun Kim, Junwoo Ha, Sangyoon Yu, Haon Park

AI总结:

本文提出 X-Teaming 进化式 M2S 框架,通过语言模型引导的进化自动发现多轮转单轮越狱模板,在 GPT-4.1 上实现 44.8% 成功率,并揭示阈值校准与跨模型评估的重要性。

摘要:

多轮到单轮(M2S)将迭代式红队测试压缩为一个结构化提示,但先前的工作依赖于少量手工编写的模板。我们提出了 X-Teaming 进化式 M2S,一个通过语言模型引导的进化来自动发现并优化 M2S 模板的框架。该系统将来自 12 个来源的智能采样与受 StrongREJECT 启发的 LLM-as-judge 相结合,并记录完全可审计的日志。通过将成功阈值设定为 θ=0.70 以保持选择压力,我们获得了五个进化世代、两个新的模板家族,以及在 GPT-4.1 上 44.8% 的总体成功率(103/230)。一个包含 2,500 次试验的平衡跨模型面板(评判器固定)表明,结构上的改进可以迁移,但效果因目标模型而异;两个模型在同一阈值下得分为零。我们还发现提示长度与得分之间存在正相关关系,这促使我们采用长度感知的评判。我们的结果表明,结构层面的搜索是获得更强单轮探测的可复现途径,并强调了阈值校准和跨模型评估的重要性。代码、配置和工件可在 https://github.com/hyunjun1121/M2S-x-teaming 获取。

英文摘要:

Multi-turn-to-single-turn (M2S) compresses iterative red-teaming into one structured prompt, but prior work relied on a handful of manually written templates. We present X-Teaming Evolutionary M2S, an automated framework that discovers and optimizes M2S templates through language-model-guided evolution. The system pairs smart sampling from 12 sources with an LLM-as-judge inspired by StrongREJECT and records fully auditable logs. Maintaining selection pressure by setting the success threshold to $θ= 0.70$, we obtain five evolutionary generations, two new template families, and 44.8% overall success (103/230) on GPT-4.1. A balanced cross-model panel of 2,500 trials (judge fixed) shows that structural gains transfer but vary by target; two models score zero at the same threshold. We also find a positive coupling between prompt length and score, motivating length-aware judging. Our results demonstrate that structure-level search is a reproducible route to stronger single-turn probes and underscore the importance of threshold calibration and cross-model evaluation. Code, configurations, and artifacts are available at https://github.com/hyunjun1121/M2S-x-teaming.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑