DRIP：通过逐token表示编辑和残差指令融合防御提示注入
DRIP: Defending Prompt Injection via Token-wise Representation Editing and Residual Instruction Fusion
AI总结：
针对传统LLM易受提示注入攻击且现有防御难以平衡实用性与安全性的问题，提出DRIP，通过去指令化数据token和残差指令融合提升鲁棒性，在基准测试中显著降低攻击成功率并保持实用性。
摘要：
大型语言模型（LLMs）正越来越多地集成到IT基础设施中，它们根据预定义指令处理用户数据。然而，传统LLMs仍易受提示注入攻击，恶意用户会将指令token注入数据以颠覆模型行为。现有防御方法训练LLMs在语义上分离数据和指令token，但仍难以（1）平衡实用性与安全性，（2）防止数据中类指令语义覆盖预期指令。我们提出DRIP，它（1）精准移除数据部分token的指令语义同时保留其数据语义，（2）在强对抗内容下稳健保留预期指令的效果。为“去指令化”数据token，DRIP引入带有轻量级表示编辑模块的数据整理与训练范式，该模块编辑数据部分类指令token的嵌入，在不损害实用性的前提下增强安全性。为确保指令不可覆盖，DRIP添加最小残差模块，降低对抗数据覆盖原始指令的能力。我们在LLaMA 8B和Mistral 7B上，针对StruQ、SecAlign、ISE和PFT四种方法，在SEP、AlpacaFarm和InjecAgent三个提示注入基准上评估DRIP。DRIP将角色分离分数提升12-49%，在自适应攻击下将攻击成功率降低66%以上，并匹配无防御模型的实用性，为提示注入鲁棒性建立了新的技术水平。
英文摘要：
Large language models (LLMs) are increasingly integrated into IT infrastructures, where they process user data according to predefined instructions. However, conventional LLMs remain vulnerable to prompt injection, where malicious users inject directive tokens into the data to subvert model behavior. Existing defenses train LLMs to semantically separate data and instruction tokens, but still struggle to (1) balance utility and security and (2) prevent instruction-like semantics in the data from overriding the intended instructions. We propose DRIP, which (1) precisely removes instruction semantics from tokens in the data section while preserving their data semantics, and (2) robustly preserves the effect of the intended instruction even under strong adversarial content. To "de-instructionalize" data tokens, DRIP introduces a data curation and training paradigm with a lightweight representation-editing module that edits embeddings of instruction-like tokens in the data section, enhancing security without harming utility. To ensure non-overwritability of instructions, DRIP adds a minimal residual module that reduces the ability of adversarial data to overwrite the original instruction. We evaluate DRIP on LLaMA 8B and Mistral 7B against StruQ, SecAlign, ISE, and PFT on three prompt-injection benchmarks (SEP, AlpacaFarm, and InjecAgent). DRIP improves role-separation score by 12-49\%, reduces attack success rate by over 66\% under adaptive attacks, and matches the utility of the undefended model, establishing a new state of the art for prompt-injection robustness.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
