arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2402.06363cs.CR
收藏

StruQ:使用结构化查询防御提示注入

StruQ: Defending Against Prompt Injection with Structured Queries

Sizhe Chen, Julien Piet, Chawin Sitawarin, David Wagner

AI总结:

针对LLM的提示注入攻击威胁,提出结构化查询方法,通过安全前端分离提示与数据,并采用新颖微调策略训练LLM仅遵循提示指令,显著提升防御能力且不损失实用性。

摘要:

大型语言模型(LLM)的最新进展催生了令人兴奋的LLM集成应用,这些应用通过利用其高级语言理解能力来执行基于文本的任务。然而,随着LLM的改进,针对它们的攻击也随之增强。提示注入攻击是一种重要威胁:它们诱骗模型偏离原始应用的指令,转而遵循用户的指示。这些攻击依赖于LLM遵循指令的能力以及无法分离提示和用户数据的能力。我们引入了结构化查询,这是一种解决该问题的通用方法。结构化查询将提示和数据分离到两个通道中。我们实现了一个支持结构化查询的系统。该系统由(1)一个安全的前端组成,它将提示和用户数据格式化为特殊格式,以及(2)一个经过特殊训练的LLM,能够从这些输入中产生高质量的输出。该LLM使用一种新颖的微调策略进行训练:我们将一个基础(非指令微调)LLM转换为结构化指令微调模型,该模型将只遵循查询提示部分中的指令。为此,我们通过在查询数据部分也包含指令的示例来扩充标准指令微调数据集,并微调模型以忽略这些指令。我们的系统显著提高了对提示注入攻击的抵抗力,而对实用性几乎没有或没有影响。我们的代码已在 https://github.com/Sizhe-Chen/StruQ 发布。

英文摘要:

Recent advances in Large Language Models (LLMs) enable exciting LLM-integrated applications, which perform text-based tasks by utilizing their advanced language understanding capabilities. However, as LLMs have improved, so have the attacks against them. Prompt injection attacks are an important threat: they trick the model into deviating from the original application's instructions and instead follow user directives. These attacks rely on the LLM's ability to follow instructions and inability to separate prompts and user data. We introduce structured queries, a general approach to tackle this problem. Structured queries separate prompts and data into two channels. We implement a system that supports structured queries. This system is made of (1) a secure front-end that formats a prompt and user data into a special format, and (2) a specially trained LLM that can produce high-quality outputs from these inputs. The LLM is trained using a novel fine-tuning strategy: we convert a base (non-instruction-tuned) LLM to a structured instruction-tuned model that will only follow instructions in the prompt portion of a query. To do so, we augment standard instruction tuning datasets with examples that also include instructions in the data portion of the query, and fine-tune the model to ignore these. Our system significantly improves resistance to prompt injection attacks, with little or no impact on utility. Our code is released at https://github.com/Sizhe-Chen/StruQ.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑