使用聚光灯技术防御间接提示注入攻击
Defending Against Indirect Prompt Injection Attacks With Spotlighting
AI总结:
本文提出聚光灯技术(spotlighting),通过转换输入提供来源信号,帮助LLM区分多输入源,有效防御间接提示注入攻击,实验显示攻击成功率从50%以上降至2%以下且对任务效能影响极小。
摘要:
大型语言模型(LLMs)虽然强大,但其构建和训练旨在处理单一文本输入。在常见应用中,可以通过将多个输入拼接成单一文本流来处理它们。然而,LLM无法区分提示的哪些部分属于不同的输入源。间接提示注入攻击利用了这一漏洞,将对抗性指令嵌入到与用户命令一起处理的不可信数据中。通常,LLM会将对抗性指令误认为需要遵循的用户命令,从而在更大系统中造成安全漏洞。我们引入了聚光灯技术(spotlighting),这是一系列提示工程技术,可用于提高LLM区分多个输入源的能力。关键见解是利用输入的转换来提供关于其来源的可靠且连续的信号。我们将聚光灯技术作为防御间接提示注入攻击的手段进行评估,发现它是一种稳健的防御方法,对底层NLP任务的最小负面影响。在使用GPT系列模型的实验中,我们发现聚光灯技术将攻击成功率从50%以上降低到2%以下,且对任务效能的影响极小。
英文摘要:
Large Language Models (LLMs), while powerful, are built and trained to process a single text input. In common applications, multiple inputs can be processed by concatenating them together into a single stream of text. However, the LLM is unable to distinguish which sections of prompt belong to various input sources. Indirect prompt injection attacks take advantage of this vulnerability by embedding adversarial instructions into untrusted data being processed alongside user commands. Often, the LLM will mistake the adversarial instructions as user commands to be followed, creating a security vulnerability in the larger system. We introduce spotlighting, a family of prompt engineering techniques that can be used to improve LLMs' ability to distinguish among multiple sources of input. The key insight is to utilize transformations of an input to provide a reliable and continuous signal of its provenance. We evaluate spotlighting as a defense against indirect prompt injection attacks, and find that it is a robust defense that has minimal detrimental impact to underlying NLP tasks. Using GPT-family models, we find that spotlighting reduces the attack success rate from greater than {50}\% to below {2}\% in our experiments with minimal impact on task efficacy.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。