arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2602.11247cs.CRcs.AI
收藏

峰值+累积:多轮LLM攻击检测的代理层评分公式

Peak + Accumulation: A Proxy-Level Scoring Formula for Multi-Turn LLM Attack Detection

J Alex Corll

AI总结:

提出一种基于峰值和累积的代理层评分公式,用于多轮LLM攻击检测,实现高召回率和低误报率。

摘要:

多轮提示注入攻击将恶意意图分布于多个对话轮次中,利用了每个轮次被独立评估的假设。尽管单轮检测已得到广泛研究,但尚未有已发表的公式能够将每轮模式得分聚合为对话层面的风险分数,且不调用LLM。我们发现直观加权平均方法的根本性缺陷:无论轮次数如何,它都会收敛到每轮得分,意味着一个持续20轮的攻击与单个可疑轮次得分相同。借鉴变化点检测(CUSUM)、贝叶斯信念更新和基于安全风险的警报方法的类比,我们提出了峰值+累积评分——一个结合峰值单轮风险、持续性比率和类别多样性。在10,654个多轮对话上进行评估——588个攻击来自WildJailbreak对抗性提示,10,066个良性对话来自WildChat——该公式在1.20%的误报率下达到90.8%的召回率,F1值为85.9%。对持续性参数的敏感性分析揭示了一个相变点,在rho ~ 0.4时,召回率增加12个百分点,而误报率增加几乎可以忽略不计。我们开源了评分算法、模式库和评估工具。

英文摘要:

Multi-turn prompt injection attacks distribute malicious intent across multiple conversation turns, exploiting the assumption that each turn is evaluated independently. While single-turn detection has been extensively studied, no published formula exists for aggregating per-turn pattern scores into a conversation-level risk score at the proxy layer -- without invoking an LLM. We identify a fundamental flaw in the intuitive weighted-average approach: it converges to the per-turn score regardless of turn count, meaning a 20-turn persistent attack scores identically to a single suspicious turn. Drawing on analogies from change-point detection (CUSUM), Bayesian belief updating, and security risk-based alerting, we propose peak + accumulation scoring -- a formula combining peak single-turn risk, persistence ratio, and category diversity. Evaluated on 10,654 multi-turn conversations -- 588 attacks sourced from WildJailbreak adversarial prompts and 10,066 benign conversations from WildChat -- the formula achieves 90.8% recall at 1.20% false positive rate with an F1 of 85.9%. A sensitivity analysis over the persistence parameter reveals a phase transition at rho ~ 0.4, where recall jumps 12 percentage points with negligible FPR increase. We release the scoring algorithm, pattern library, and evaluation harness as open source.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑