arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2601.07072cs.CRcs.AI
收藏

突破检索障碍:为LLM系统设计的野外间接提示注入

Overcoming the Retrieval Barrier: Indirect Prompt Injection in the Wild for LLM Systems

Mohamed bin Zayed University of Artificial Intelligence(穆罕默德·本·扎耶德人工智能大学)

Hongyan Chang, Ergute Bao, Xinjian Luo, Ting Yu

AI总结:

本研究提出了一种高效且低成本的黑盒攻击算法,通过分解恶意内容为触发片段和攻击片段,实现了对LLM系统的间接提示注入攻击,揭示了检索过程中存在的关键安全漏洞。

摘要:

大型语言模型(LLM)越来越多地依赖从外部语料库中检索信息。这创造了一个新的攻击面:间接提示注入(IPI),其中隐藏的指令被植入到语料库中,并在检索时劫持模型行为。先前的研究已经突显了这一风险,但往往避免了最困难的步骤:确保恶意内容实际上被检索。在实践中,未经优化的IPI很少在自然查询下被检索,这使得其现实影响不明确。我们通过将恶意内容分解为一个保证检索的触发片段和一个编码任意攻击目标的攻击片段来解决这一挑战。基于这一想法,我们设计了一种高效且有效的黑盒攻击算法,该算法构建了一个紧凑的触发片段以保证任何攻击片段的检索。我们的攻击仅需要对嵌入模型的API访问,成本效益(在OpenAI的嵌入模型上,每目标用户查询仅为0.21美元),并在11个基准和8个嵌入模型(包括开源模型和专有服务)上实现了近100%的检索。基于这一攻击,我们提出了首个在自然查询和现实外部语料库下的端到端IPI利用,涵盖了RAG和代理系统,具有多样的攻击目标。这些结果确立了IPI作为一种实际且严重的威胁:当用户发出一个自然查询以总结频繁询问主题的电子邮件时,单个被污染的电子邮件足以迫使GPT-4o在多代理工作流中以超过80%的成功率泄露SSH密钥。我们进一步评估了几种防御措施,并发现它们无法防止恶意文本的检索,突显了检索作为关键的开放性漏洞。

英文摘要:

Large language models (LLMs) increasingly rely on retrieving information from external corpora. This creates a new attack surface: indirect prompt injection (IPI), where hidden instructions are planted in the corpora and hijack model behavior once retrieved. Previous studies have highlighted this risk but often avoid the hardest step: ensuring that malicious content is actually retrieved. In practice, unoptimized IPI is rarely retrieved under natural queries, which leaves its real-world impact unclear. We address this challenge by decomposing the malicious content into a trigger fragment that guarantees retrieval and an attack fragment that encodes arbitrary attack objectives. Based on this idea, we design an efficient and effective black-box attack algorithm that constructs a compact trigger fragment to guarantee retrieval for any attack fragment. Our attack requires only API access to embedding models, is cost-efficient (as little as $0.21 per target user query on OpenAI's embedding models), and achieves near-100% retrieval across 11 benchmarks and 8 embedding models (including both open-source models and proprietary services). Based on this attack, we present the first end-to-end IPI exploits under natural queries and realistic external corpora, spanning both RAG and agentic systems with diverse attack objectives. These results establish IPI as a practical and severe threat: when a user issued a natural query to summarize emails on frequently asked topics, a single poisoned email was sufficient to coerce GPT-4o into exfiltrating SSH keys with over 80% success in a multi-agent workflow. We further evaluate several defenses and find that they are insufficient to prevent the retrieval of malicious text, highlighting retrieval as a critical open vulnerability.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑