arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2403.02817cs.CR
收藏

AI蠕虫来袭:释放针对GenAI驱动的应用的零点击蠕虫

Here Comes The AI Worm: Unleashing Zero-click Worms that Target GenAI-Powered Applications

Stav Cohen, Ron Bitton, Ben Nassi

AI总结:

本文提出了一种名为Morris-II的零点击AI蠕虫,通过对抗性自复制提示词在基于RAG的GenAI应用生态中引发链式间接提示注入,窃取机密数据;同时提出Virtual Donkey护栏,以低延迟和高准确率有效检测并阻止该蠕虫传播。

摘要:

在本文中,我们展示了当GenAI驱动的应用之间的通信依赖于基于RAG的推理时,攻击者可以发起一种类似计算机蠕虫的链式反应,我们称之为Morris-II。这是通过制作一个对抗性自复制提示词来实现的,该提示词在生态系统内触发一系列间接提示注入,并迫使每个受影响的应用执行恶意操作,从而攻陷其他应用的RAG。我们在一个由GenAI驱动的电子邮件助手组成的GenAI生态系统中,评估了该蠕虫在创建机密用户数据提取链方面的性能,并分析了蠕虫的性能如何受到上下文大小、所使用的对抗性自复制提示词、所采用的嵌入算法的类型和大小以及传播中的跳数的影响。最后,我们引入了Virtual Donkey,这是一种护栏,旨在以最小的延迟、高准确率和低误报率来检测和阻止Morris-II的传播。我们评估了该护栏的性能,并表明它产生了1.0的完美真阳性率和0.015的误报率,并且对分布外蠕虫具有鲁棒性,这些蠕虫包含未见过的越狱命令、不同的电子邮件数据集和各种蠕虫用例。

英文摘要:

In this paper, we show that when the communication between GenAI-powered applications relies on RAG-based inference, an attacker can initiate a computer worm-like chain reaction that we call Morris-II. This is done by crafting an adversarial self-replicating prompt that triggers a cascade of indirect prompt injections within the ecosystem and forces each affected application to perform malicious actions and compromise the RAG of additional applications. We evaluate the performance of the worm in creating a chain of confidential user data extraction within a GenAI ecosystem of GenAI-powered email assistants and analyze how the performance of the worm is affected by the size of the context, the adversarial self-replicating prompt used, the type and size of the embedding algorithm employed, and the number of hops in the propagation. Finally, we introduce the Virtual Donkey, a guardrail intended to detect and prevent the propagation of Morris-II with minimal latency, high accuracy, and a low false-positive rate. We evaluate the guardrail's performance and show that it yields a perfect true-positive rate of 1.0 with a false-positive rate of 0.015, and is robust against out-of-distribution worms, consisting of unseen jailbreaking commands, a different email dataset, and various worm usecases.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑