arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2410.09102cs.LGcs.AIcs.CLcs.CR
收藏

指令片段嵌入:利用指令层次结构提升LLM安全性

Instructional Segment Embedding: Improving LLM Safety with Instruction Hierarchy

Princeton University(普林斯顿大学);Zoom Video Communications(Zoom视频通讯)

Tong Wu, Shujian Zhang, Kaiqiang Song, Silei Xu, Sanqiang Zhao, Ravi Agrawal, Sathish Reddy Indurthi, Chong Xiang, Prateek Mittal, Wenxuan Zhou

AI总结:

针对LLM缺乏指令层次结构导致易受恶意提示攻击的问题,提出指令片段嵌入(ISE)技术,在架构层面将指令优先级直接嵌入模型,显著提升了模型的安全性与指令遵循能力。

摘要:

大型语言模型(LLMs)容易受到提示注入、提示提取和有害请求等安全威胁。这些漏洞的一个主要原因是缺乏指令层次结构。现代LLM架构对所有输入一视同仁,未能区分并优先处理各种类型的指令,如系统消息、用户提示和数据。因此,低优先级的用户提示可能会覆盖更关键的系统指令(包括安全协议)。现有实现指令层次结构的方法(如分隔符和基于指令的训练)并未在架构层面解决此问题。我们受BERT启发,向现代大型语言模型引入了指令片段嵌入(ISE)技术,将指令优先级信息直接嵌入模型中。该方法使模型能够显式地区分和优先处理各种指令类型,显著提升了针对试图覆盖优先级规则的恶意提示的安全性。我们在Structured Query和Instruction Hierarchy基准上的实验表明,平均鲁棒准确率分别提升了15.75%和18.68%。此外,在AlpacaEval上评估发现,指令遵循能力提升了4.1%。总体而言,我们的方法为增强LLM架构的安全性和有效性提供了有前景的方向。

英文摘要:

Large Language Models (LLMs) are susceptible to security and safety threats, such as prompt injection, prompt extraction, and harmful requests. One major cause of these vulnerabilities is the lack of an instruction hierarchy. Modern LLM architectures treat all inputs equally, failing to distinguish between and prioritize various types of instructions, such as system messages, user prompts, and data. As a result, lower-priority user prompts may override more critical system instructions, including safety protocols. Existing approaches to achieving instruction hierarchy, such as delimiters and instruction-based training, do not address this issue at the architectural level. We introduce the Instructional Segment Embedding (ISE) technique, inspired by BERT, to modern large language models, which embeds instruction priority information directly into the model. This approach enables models to explicitly differentiate and prioritize various instruction types, significantly improving safety against malicious prompts that attempt to override priority rules. Our experiments on the Structured Query and Instruction Hierarchy benchmarks demonstrate an average robust accuracy increase of up to 15.75% and 18.68%, respectively. Furthermore, we observe an improvement in instruction-following capability of up to 4.1% evaluated on AlpacaEval. Overall, our approach offers a promising direction for enhancing the safety and effectiveness of LLM architectures.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑