arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2409.14729cs.CRcs.AI
收藏

PROMPTFUZZ:利用模糊测试技术对 LLM 中的提示注入进行鲁棒性测试

PROMPTFUZZ: Harnessing Fuzzing Techniques for Robust Testing of Prompt Injection in LLMs

Northwestern University(西北大学);University of the Chinese Academy of Sciences(中国科学院大学)

Jiahao Yu, Yangguang Shao, Hanwen Miao, Junzheng Shi

AI总结:

本文提出 PROMPTFUZZ 框架,利用模糊测试技术系统评估大型语言模型抵御提示注入攻击的鲁棒性,在真实竞赛中取得前 0.14% 的成绩,并构建数据集微调模型,揭示了持续进行鲁棒性测试的重要性。

摘要:

大型语言模型(LLMs)因其生成类人文本的强大能力而在各种应用中得到广泛使用。然而,提示注入攻击通过用恶意提示覆盖模型的原始指令来操纵生成的文本,引发了对 LLMs 安全性和可靠性的严重担忧。确保 LLMs 免受此类攻击的鲁棒性对于其在现实世界应用中的部署至关重要,尤其是在关键任务中。在本文中,我们提出了 PROMPTFUZZ,这是一种新颖的测试框架,它利用模糊测试技术来系统评估 LLMs 抵御提示注入攻击的鲁棒性。受软件模糊测试的启发,PROMPTFUZZ 选择有潜力的种子提示,并生成多样化的提示注入集合,以评估目标 LLM 的抵抗力。PROMPTFUZZ 分两个阶段运行:准备阶段,涉及选择有潜力的初始种子并收集少样本示例;聚焦阶段,使用收集到的示例生成多样化、高质量的提示注入。通过使用 PROMPTFUZZ,我们可以发现 LLMs 中更多的漏洞,甚至是那些具有强防御提示的 LLMs。通过在真实世界竞赛中部署 PROMPTFUZZ 生成的攻击提示,我们在 2 小时内获得了 4000 多名参与者中的第 7 名(前 0.14%)。此外,我们构建了一个数据集来微调 LLMs,以增强其抵御提示注入攻击的鲁棒性。虽然微调后的模型显示出提升的鲁棒性,但 PROMPTFUZZ 仍能继续识别漏洞,这凸显了对 LLMs 进行鲁棒性测试的重要性。我们的工作强调了对有效测试工具的关键需求,并为评估和改进 LLMs 抵御提示注入攻击的鲁棒性提供了一个实用的框架。

英文摘要:

Large Language Models (LLMs) have gained widespread use in various applications due to their powerful capability to generate human-like text. However, prompt injection attacks, which involve overwriting a model's original instructions with malicious prompts to manipulate the generated text, have raised significant concerns about the security and reliability of LLMs. Ensuring that LLMs are robust against such attacks is crucial for their deployment in real-world applications, particularly in critical tasks. In this paper, we propose PROMPTFUZZ, a novel testing framework that leverages fuzzing techniques to systematically assess the robustness of LLMs against prompt injection attacks. Inspired by software fuzzing, PROMPTFUZZ selects promising seed prompts and generates a diverse set of prompt injections to evaluate the target LLM's resilience. PROMPTFUZZ operates in two stages: the prepare phase, which involves selecting promising initial seeds and collecting few-shot examples, and the focus phase, which uses the collected examples to generate diverse, high-quality prompt injections. Using PROMPTFUZZ, we can uncover more vulnerabilities in LLMs, even those with strong defense prompts. By deploying the generated attack prompts from PROMPTFUZZ in a real-world competition, we achieved the 7th ranking out of over 4000 participants (top 0.14%) within 2 hours. Additionally, we construct a dataset to fine-tune LLMs for enhanced robustness against prompt injection attacks. While the fine-tuned model shows improved robustness, PROMPTFUZZ continues to identify vulnerabilities, highlighting the importance of robust testing for LLMs. Our work emphasizes the critical need for effective testing tools and provides a practical framework for evaluating and improving the robustness of LLMs against prompt injection attacks.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑