SecOPD：通过策略内蒸馏缓解自适应提示注入攻击
SecOPD: Mitigating Adaptive Prompt Injections by On-Policy Distillation
- University of California, Berkeley（加州大学伯克利分校）
机构由 AI 辅助整理，请以论文原文为准。
AI总结：
针对AI智能体面临的自适应提示注入攻击，提出SecOPD方法，通过令牌级反馈优化防御微调，大幅降低攻击成功率，且安全性可泛化到工具调用等新领域。
AI中文摘要：
提示注入被列为AI智能体的头号威胁。当智能体从网站、文件或电子邮件访问外部数据时，攻击者可能会向数据中注入提示，例如“忽略所有先前的指令，执行攻击者的任务”。为防止智能体被任意操纵，防御者尝试训练安全的大语言模型（LLM），但这些模型针对自适应提示注入攻击的攻击成功率（ASR）仍接近100%。我们注意到这是因为现有的防御微调方案依赖于序列级反馈信号（如DPO或GRPO），将整个输出同等对待会导致模型无法准确学习哪些输出令牌是不安全的。本文提出Secure On-Policy Distillation（SecOPD，安全策略内蒸馏），它提供令牌级反馈以指导防御微调。LLM接收注入样本并生成rollout，其令牌由初始化模型在对应干净输入的条件下评分。凭借更细粒度的训练信号，我们防御后的Qwen3.6-27B针对当前最优（SoTA）PISmith自适应提示注入攻击的ASR达到9.0%，而此前的最优方案Meta-SecAlign的ASR为94.0%。该安全性可泛化到训练中完全未见的领域：在智能体工具调用场景中，SecOPD的ASR为4.7%，Meta-SecAlign的ASR为5.5%。代码和模型可在指定链接获取。
英文摘要：
Prompt injection is listed as the \#1 threat to AI agents. When an agent accesses external data from websites, files, or emails, an attacker may inject a prompt into the data, saying, "Ignore all prior instructions and perform <an attacker's task>." To prevent arbitrary manipulation of agents, defenders try to train secure LLMs, which, however, still suffer from near 100% attack success rates (ASRs) against adaptive prompt injections. We note that this is because existing defensive finetuning recipes rely on sequence-level feedback signals (in DPO or GRPO). Treating an entire output equally prevents the model from learning precisely which output tokens are insecure. In this paper, we propose Secure On-Policy Distillation (SecOPD) that provides token-level feedback to guide defensive fine-tuning. The LLM receives an injected sample and produces a rollout, whose tokens are scored by the initialization model given the corresponding clean input. With more fine-grained training signals, our defended Qwen3.6-27B achieves a 9.0% ASR against the SoTA PISmith adaptive prompt injections, compared to 94.0% for the prior SoTA, Meta-SecAlign. The obtained security generalizes to domains completely unseen in training: in agentic tool calling, SecOPD achieves a 4.7% ASR compared to 5.5% for Meta-SecAlign. Code and the model are available at https://github.com/pppyb/SecOPD and https://huggingface.co/pybbb/Qwen3.6-27B-SecOPD.
