arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2604.28129cs.CRcs.AI
收藏

潜在对抗检测:适应性探测LLM激活以多轮攻击检测

Latent Adversarial Detection: Adaptive Probing of LLM Activations for Multi-Turn Attack Detection

Mountain View, CA(山景城,加利福尼亚州)

Prashant Kulkarni

AI总结:

本文通过分析LLM激活层的轨迹特征,提出对抗性不稳定性概念,提升多轮攻击检测精度,验证了模型家族间的信号一致性及数据分布对泛化能力的影响。

摘要:

多轮提示注入攻击遵循信任建立、转向、升级的已知路径,但文本级防御无法检测隐蔽攻击,其中单轮看似无害。我们发现该攻击路径在模型残差流中留下激活层签名:每个阶段转移使激活移动,产生远超良性对话的总路径长度。我们称此为对抗性不稳定性。五种标量轨迹特征捕获此信号,使对话级检测从76.2%提升至93.8%。该信号在四个模型家族(24B-70B)中复现;探测器特定于模型且不跨架构转移。泛化依赖于数据源:留一数据源验证显示合成、LMSYS-Chat-1M和SafeDialBench捕获不同的攻击分布,当真实世界LMSYS数据分布被训练时,检测率达47-71%。三源训练在混合测试集上实现89.4%检测率,假阳性率2.4%。此外,三种阶段的对话级标签(良性/转向/对抗)对检测至关重要:二元对话级标签导致50-59%假阳性。这些结果证明对抗性不稳定性是可靠的激活层信号,并明确了实际部署的数据需求。

英文摘要:

Multi-turn prompt injection follows a known attack path -- trust-building, pivoting, escalation but text-level defenses miss covert attacks where individual turns appear benign. We show this attack path leaves an activation-level signature in the model's residual stream: each phase shift moves the activation, producing a total path length far exceeding benign conversations. We call this adversarial restlessness. Five scalar trajectory features capturing this signal lift conversation-level detection from 76.2% to 93.8% on synthetic held-out data. The signal replicates across four model families (24B-70B); probes are model-specific and do not transfer across architectures. Generalization is source-dependent: leave-one-source-out evaluation shows each of synthetic, LMSYS-Chat-1M, and SafeDialBench captures distinct attack distributions, with detection on real-world LMSYS reaching 47-71% when its distribution is represented in training. Combined three-source training achieves 89.4% detection at 2.4% false positive rate on a held-out mixed set. We further show that three-phase turn-level labels(benign/pivoting/adversarial) unique to our synthetic dataset are essential: binary conversation-level labels produce 50-59% false positives. These results establish adversarial restlessness as a reliable activation-level signal and characterize the data requirements for practical deployment.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑