arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2407.09164cs.CRcs.AI
收藏

ShadowCode:面向代码大语言模型的(自动)外部提示注入攻击研究

ShadowCode: Towards (Automatic) External Prompt Injection Attack against Code LLMs

State Key Laboratory of Blockchain and Data Security, Zhejiang University(区块链与数据安全国家重点实验室,浙江大学);Hangzhou High-Tech Zone (Binjiang) Institute of Blockchain and Data Security, Hangzhou(杭州高新区(滨江)区块链与数据安全研究院,杭州);College of Computing and Data Science, Nanyang Technological University(计算与数据科学学院,南洋理工大学)

Yuchen Yang, Yiming Li, Hongwei Yao, Bingrun Yang, Yiling He, Tianwei Zhang, Dacheng Tao, Zhan Qin

AI总结:

本文提出ShadowCode方法,通过自动生成非功能性诱导扰动注入代码上下文,在不控制模型训练的情况下实现对代码大语言模型的有效隐蔽外部提示注入攻击,在多个开源和商业模型上取得超过90%的攻击成功率。

摘要:

近年来,代码导向的大语言模型(Code LLMs)在编程任务中得到了广泛应用。尽管它们在部署上取得了成功,但其安全性研究却远远滞后。本文提出了一种新的攻击范式:针对代码大语言模型的(自动)外部提示注入攻击,攻击者生成简洁的、非功能性的诱导扰动,并将其注入受害者的代码上下文中。这些诱导扰动可以通过常用的依赖项(例如软件包或RAG的知识库)进行传播,从而在代码补全过程中操纵代码大语言模型以实现恶意目标。与现有攻击相比,该方法更具现实性和威胁性:与后门攻击不同,它不需要控制模型的训练过程,并且能够实现对抗性攻击难以达成的特定恶意目标。此外,我们提出了ShadowCode,这是一种简单而有效的方法,能够基于代码模拟自动生成诱导扰动,以实现有效且隐蔽的外部提示注入。ShadowCode通过模拟真实的代码上下文来设计其扰动优化目标,并采用带有两个增强模块的贪心优化方法:前向推理增强和基于关键词的扰动设计。我们在13个不同的恶意目标上评估了我们的方法,生成了涵盖三种流行编程语言的31个威胁案例。实验结果表明,ShadowCode在所有威胁案例中成功攻击了三个具有代表性的开源代码大语言模型(攻击成功率最高达97.9%)和两个主流的集成代码大语言模型的商业应用(攻击成功率超过90%),且仅使用了一个12个token的非功能性诱导扰动。代码可在https://github.com/LianPing-cyber/ShadowCodeEPI获取。

英文摘要:

Recent advancements have led to the widespread adoption of code-oriented large language models (Code LLMs) for programming tasks. Despite their success in deployment, their security research is left far behind. This paper introduces a new attack paradigm: (automatic) external prompt injection against Code LLMs, where attackers generate concise, non-functional induced perturbations and inject them within a victim's code context. These induced perturbations can be disseminated through commonly used dependencies (e.g., packages or RAG's knowledge base), manipulating Code LLMs to achieve malicious objectives during the code completion process. Compared to existing attacks, this method is more realistic and threatening: it does not necessitate control over the model's training process, unlike backdoor attacks, and can achieve specific malicious objectives that are challenging for adversarial attacks. Furthermore, we propose ShadowCode, a simple yet effective method that automatically generates induced perturbations based on code simulation to achieve effective and stealthy external prompt injection. ShadowCode designs its perturbation optimization objectives by simulating realistic code contexts and employs a greedy optimization approach with two enhancement modules: forward reasoning enhancement and keyword-based perturbation design. We evaluate our method across 13 distinct malicious objectives, generating 31 threat cases spanning three popular programming languages. Our results demonstrate that ShadowCode successfully attacks three representative open-source Code LLMs (achieving up to a 97.9% attack success rate) and two mainstream commercial Code LLM-integrated applications (with over 90% attack success rate) across all threat cases, using only a 12-token non-functional induced perturbation. The code is available at https://github.com/LianPing-cyber/ShadowCodeEPI.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑