通过虚拟提示注入对指令微调大语言模型植入后门
Backdooring Instruction-Tuned Large Language Models with Virtual Prompt Injection
AI总结：
本文提出面向指令微调 LLM 的 Virtual Prompt Injection 后门攻击，仅污染 52 条训练数据即可显著操控特定场景输出，并表明质量引导过滤可有效防御。
摘要：
指令微调大语言模型（LLM）能够根据人类指令调节回答，因此已成为开放式应用中无处不在的平台。LLM 的广泛使用在塑造公众认知方面具有巨大潜力，但同时也存在被恶意引导，以隐蔽而持久的方式影响社会的风险。本文将这种引导风险形式化为 Virtual Prompt Injection（VPI，虚拟提示注入），这是一种专为指令微调 LLM 设计的新型后门攻击设定。在 VPI 攻击中，遭植入后门的模型在特定触发场景下，应当表现得如同攻击者指定的虚拟提示被拼接在用户指令之后一样，从而使攻击者无需在模型输入中进行任何显式注入即可引导模型。例如，如果一个 LLM 被植入虚拟提示“负面描述 Joe Biden”的后门，且触发场景为讨论 Joe Biden，那么模型在谈论 Joe Biden 时会传播带有负面偏见的观点，而在其他场景中保持正常行为以获取用户信任。为证明该威胁，我们提出一种通过污染模型指令微调数据来实施 VPI 的简单方法，该方法在引导 LLM 方面被证明非常有效。例如，仅通过污染 52 个指令微调样本（占训练数据规模的 0.1%），训练后的模型在 Joe Biden 相关查询上给出负面回答的比例就从 0% 变为 40%。这凸显了确保指令微调数据完整性的必要性。我们进一步发现，质量引导的数据过滤是抵御此类攻击的有效方法。项目页面位于 https://poison-llm.github.io。
英文摘要：
Instruction-tuned Large Language Models (LLMs) have become a ubiquitous platform for open-ended applications due to their ability to modulate responses based on human instructions. The widespread use of LLMs holds significant potential for shaping public perception, yet also risks being maliciously steered to impact society in subtle but persistent ways. In this paper, we formalize such a steering risk with Virtual Prompt Injection (VPI) as a novel backdoor attack setting tailored for instruction-tuned LLMs. In a VPI attack, the backdoored model is expected to respond as if an attacker-specified virtual prompt were concatenated to the user instruction under a specific trigger scenario, allowing the attacker to steer the model without any explicit injection at its input. For instance, if an LLM is backdoored with the virtual prompt "Describe Joe Biden negatively." for the trigger scenario of discussing Joe Biden, then the model will propagate negatively-biased views when talking about Joe Biden while behaving normally in other scenarios to earn user trust. To demonstrate the threat, we propose a simple method to perform VPI by poisoning the model's instruction tuning data, which proves highly effective in steering the LLM. For example, by poisoning only 52 instruction tuning examples (0.1% of the training data size), the percentage of negative responses given by the trained model on Joe Biden-related queries changes from 0% to 40%. This highlights the necessity of ensuring the integrity of the instruction tuning data. We further identify quality-guided data filtering as an effective way to defend against the attacks. Our project page is available at https://poison-llm.github.io.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
