发表机构
Vulcan Research, AIFT（Vulcan研究，AIFT）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
研究针对大语言模型安全评估问题，提出AMT-X框架，将攻击设为多阶段状态机，用多角色评审团取代单评判评分，在六个前沿受害者模型和七个审核子类别测试中，不同门控下攻击成功率有显著差异，揭示了部分与完全可操作危害的差距。
AI 中文摘要
大语言模型的安全评估很大程度上依赖单轮攻击数据集和单评判评分，这低估了自适应多轮对手的风险，且单一成功率无法区分部分可操作输出和完整操作细节输出。我们提出了AMT-X（自适应多轮利用），这是一个阶段结构化多轮红队测试框架。与之前依赖临时升级或自由形式逐目标计划的多轮攻击不同，AMT-X将攻击视为由受害者语义信号驱动的明确、可重复的多阶段状态机，并用多角色评审团取代单评判评分，其阶段条件清单根据可操作危害来判定成功。在六个前沿受害者模型和七个审核子类别中，AMT-X在宽松分数阈值下总体攻击成功率为97.6%-100%，但在要求完整、真实和操作细节的更严格门控下为66.7%-78.6%，部分和完全可操作危害之间差距高达33个百分点。
英文摘要
Safety evaluation of large language models (LLMs) relies largely on single-turn attack datasets and single-judge scoring, underestimating risk from adaptive multi-turn adversaries and reporting a single success rate that does not separate partially actionable outputs from those carrying complete operational detail. We propose AMT-X (Adaptive Multi-Turn Exploitation), a phase-structured multi-turn red-teaming framework. Unlike prior multi-turn attacks that rely on ad hoc escalation or free-form per-goal plans, AMT-X casts the attack as an explicit, reproducible multi-phase state machine driven by semantic signals from the victim, and replaces single-judge scoring with a multi-role jury whose phase-conditioned checklists gate success on actionable harm. Across six frontier victim models (queried under their default safety alignment, without added moderation layers) and seven Moderation sub-categories, AMT-X attains overall attack success rates of 97.6-100% under a lenient score threshold, but 66.7-78.6% under a stricter gate requiring complete, real, and operational detail: a gap of up to 33 percentage points between partially and fully actionable harm.
CommentsA reference implementation is available at https://github.com/VulcanLab/amt-x
