指令层级：训练 LLM 优先处理特权指令
The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructions
AI总结：
针对LLM因系统提示与不可信输入同优先级而易受攻击的漏洞，提出指令层级框架及数据生成方法，训练GPT-3.5选择性忽略低特权指令，大幅提升对未见攻击的鲁棒性且基本不损失标准能力。
摘要：
如今的 LLM 容易受到提示注入、越狱和其他攻击，这些攻击允许对手用自己的恶意提示覆盖模型的原始指令。在这项工作中，我们认为这些攻击背后的主要漏洞之一是，LLM 通常将系统提示（例如，来自应用程序开发者的文本）视为与来自不受信任的用户和第三方的文本具有相同的优先级。为了解决这个问题，我们提出了一种指令层级，明确定义当不同优先级的指令发生冲突时模型应如何表现。然后，我们提出了一种数据生成方法来展示这种分层指令遵循行为，该方法教导 LLM 选择性地忽略较低特权的指令。我们将此方法应用于 GPT-3.5，结果表明它大幅提高了鲁棒性——即使对于训练期间未见的攻击类型——同时对标准能力的影响微乎其微。
英文摘要：
Today's LLMs are susceptible to prompt injections, jailbreaks, and other attacks that allow adversaries to overwrite a model's original instructions with their own malicious prompts. In this work, we argue that one of the primary vulnerabilities underlying these attacks is that LLMs often consider system prompts (e.g., text from an application developer) to be the same priority as text from untrusted users and third parties. To address this, we propose an instruction hierarchy that explicitly defines how models should behave when instructions of different priorities conflict. We then propose a data generation method to demonstrate this hierarchical instruction following behavior, which teaches LLMs to selectively ignore lower-privileged instructions. We apply this method to GPT-3.5, showing that it drastically increases robustness -- even for attack types not seen during training -- while imposing minimal degradations on standard capabilities.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
