哪种防御措施应对哪种威胁？归因OWASP-LLM-Top-10覆盖及其在释义下的脆弱性
Which Defense Closes Which Threat? Attributing OWASP-LLM-Top-10 Coverage and Its Brittleness Under Paraphrasing
AI总结：
本文通过归因分析，测量了不同防御家族（拒绝过滤、预算控制等）对OWASP-LLM-Top-10威胁的覆盖情况，并揭示了拒绝防御在释义攻击下的脆弱性。
摘要：
生产级LLM应用堆叠了多种防御家族——拒绝短语过滤器、令牌预算控制、模型白名单、速率限制、工具注册认证——然而现有的攻防模拟（BAS）基准报告单一的总体覆盖数字，隐藏了哪个家族应对哪种威胁。我们测量归因。我们将四个OWASP-LLM-Top-10感知的智能体添加到一个21智能体的基线扫描器中，并针对四个合成LLM端点的格点：$L_0$（无防御）、$L_1$（仅拒绝）、$L_2$（仅预算）和$L_3$（全栈）。$L_1$和$L_2$是兄弟单轴消融，互不为子集；$L_3$是它们的并集加上工具注册认证和凭证清洗。在$N=10$次重复中，每个OWASP的发现计数清晰：仅拒绝消除所有LLM01（越狱）和LLM07（系统提示泄露）发现；仅预算通过终止多步序列消除所有LLM02（敏感信息泄露）和LLM10（无限制消耗）发现；LLM06（过度代理）需要全栈。我们探测释义下的脆弱性：使用300个Gemini生成的释义（在60模板脆弱性语料库上$K=5$），$L_1$拒绝阻断率在LLM01上下降15个百分点，在LLM07上下降25个百分点。第五个目标$L_4$-real，将存根后端替换为Gemini-2.5-flash，使用相同的$L_3$正则表达式，并与$L_1$完全匹配，表明除了正则表达式外没有可测量的对齐贡献（不是关于对齐的一般性声明）。预算控制没有下降（在扣除速率限制下限后为0个百分点）。一个通过静态基准的拒绝白名单可以被LLM驱动的释义器击败而不改变攻击意图；预算控制抵抗相同的变异。
英文摘要：
Production LLM applications stack several defense families -- refusal-phrase filters, token-budget controls, model allowlists, rate limits, tool-registry authentication -- yet existing breach-and-attack-simulation (BAS) benchmarks report a single aggregate coverage number, hiding which family closes which threat. We measure attribution. We add four OWASP-LLM-Top-10-aware agents to a 21-agent baseline scanner and target a lattice of four synthetic LLM endpoints: $L_0$ (no defenses), $L_1$ (refusal-only), $L_2$ (budget-only), and $L_3$ (full stack). $L_1$ and $L_2$ are sibling single-axis ablations, not subsets of each other; $L_3$ is their union plus tool-registry authentication and credential scrubbing. Across $N=10$ replications, the per-OWASP finding count is clean: refusal alone removes all LLM01 (jailbreak) and LLM07 (system-prompt leakage) findings; budget alone removes all LLM02 (sensitive-info disclosure) and LLM10 (unbounded consumption) findings by terminating multi-step sequences; LLM06 (excessive agency) requires the full stack. We probe brittleness under paraphrasing: with 300 Gemini-generated paraphrases ($K=5$ over a 60-template brittleness corpus), $L_1$ refusal block rate falls 15 pp on LLM01 and 25 pp on LLM07. A fifth target, $L_4$-real, swaps the stub backend for Gemini-2.5-flash behind the same $L_3$ regex and matches $L_1$ exactly, indicating no measurable alignment contribution beyond the regex (not a general claim about alignment). Budget controls show no drop (0 pp once the rate-limit floor is factored out). A refusal whitelist that clears a static benchmark can be defeated by an LLM-driven paraphraser without changing attack intent; a budget control resists the same mutation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
