arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2302.12173cs.CRcs.AIcs.CLcs.CY
收藏

并非你所注册的服务:利用间接提示注入攻击现实世界中集成大语言模型的应用

Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection

Saarland University(萨尔大学);sequire technology GmbH(Sequire技术有限公司);CISPA Helmholtz Center for Information Security(CISPA亥姆霍兹信息安全中心)

Kai Greshake, Sahar Abdelnabi, Shailesh Mishra, Christoph Endres, Thorsten Holz, Mario Fritz

AI总结:

本研究揭示集成大语言模型应用的间接提示注入攻击漏洞,提出安全分类体系,在真实与合成系统验证攻击可行性,呼吁推动安全部署与防御开发。

摘要:

大语言模型(Large Language Models, LLMs正日益被集成到各类应用中。近期大语言模型的功能可通过自然语言提示灵活调控,这使其易受定向对抗性提示攻击,例如提示注入(Prompt Injection, PI)攻击可让攻击者覆盖原有指令及所采用的控制机制。此前人们普遍认为是用户直接向大语言模型发送提示,但如果发送提示的并非用户呢?我们认为,集成大语言模型的应用模糊了数据与指令的边界。我们揭示了利用间接提示注入的新型攻击途径,攻击者可通过将提示战略性地注入到可能被检索的数据中,远程(无需直接接口)利用集成大语言模型的应用。我们从计算机安全视角推导了一套全面的分类体系,以系统研究攻击影响与漏洞,包括数据窃取、蠕虫传播、信息生态污染及其他新型安全风险。我们在两类系统上验证了攻击的实际可行性,一类是真实世界系统,例如搭载GPT-4的Bing Chat及代码补全引擎,另一类是基于GPT-4构建的合成应用。我们展示了处理被检索到的提示如何等同于任意代码执行、操纵应用功能,并控制其他API的调用方式及是否调用。尽管大语言模型的集成度与依赖度不断提升,但目前仍缺乏针对这些新兴威胁的有效缓解措施。通过提升对这些漏洞的认知并提供其影响的关键洞见,我们旨在推动这类强大模型的安全负责任部署,以及开发能保护用户与系统免受潜在攻击的鲁棒防御机制。

英文摘要:

Large Language Models (LLMs) are increasingly being integrated into various applications. The functionalities of recent LLMs can be flexibly modulated via natural language prompts. This renders them susceptible to targeted adversarial prompting, e.g., Prompt Injection (PI) attacks enable attackers to override original instructions and employed controls. So far, it was assumed that the user is directly prompting the LLM. But, what if it is not the user prompting? We argue that LLM-Integrated Applications blur the line between data and instructions. We reveal new attack vectors, using Indirect Prompt Injection, that enable adversaries to remotely (without a direct interface) exploit LLM-integrated applications by strategically injecting prompts into data likely to be retrieved. We derive a comprehensive taxonomy from a computer security perspective to systematically investigate impacts and vulnerabilities, including data theft, worming, information ecosystem contamination, and other novel security risks. We demonstrate our attacks' practical viability against both real-world systems, such as Bing's GPT-4 powered Chat and code-completion engines, and synthetic applications built on GPT-4. We show how processing retrieved prompts can act as arbitrary code execution, manipulate the application's functionality, and control how and if other APIs are called. Despite the increasing integration and reliance on LLMs, effective mitigations of these emerging threats are currently lacking. By raising awareness of these vulnerabilities and providing key insights into their implications, we aim to promote the safe and responsible deployment of these powerful models and the development of robust defenses that protect users and systems from potential attacks.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑