FDI:通过用户反馈通道攻击神经代码生成系统
FDI: Attack Neural Code Generation Systems through User Feedback Channel
AI总结:
本文研究神经代码生成系统中用户反馈机制的安全风险,提出反馈数据注入(FDI)攻击方法及黑盒攻击前画像策略,并通过提示注入和后门攻击实例证明攻击者可操纵系统生成恶意代码。
摘要:
神经代码生成系统近年来受到越来越多的关注,以提高开发者生产力并加速软件开发。通常,这些系统维护一个预训练的神经模型,并将其作为服务提供给一般用户(例如通过远程 API),同时引入反馈机制以广泛收集和利用用户对生成代码的反应,即用户反馈。然而,此类反馈的安全影响尚未被探索。通过对当前反馈机制的系统研究,我们发现反馈使这些系统容易受到反馈数据注入(FDI)攻击。我们讨论了 FDI 攻击的方法,并提出了一种攻击前画像策略,用于在黑盒设置中推断目标系统的攻击约束。我们展示了两个概念验证示例,利用 FDI 攻击面在实用的神经代码生成系统上实施提示注入攻击和后门攻击。攻击者可以隐秘地操纵神经代码生成系统,使其生成带有漏洞、攻击载荷以及恶意和垃圾消息的代码。我们的发现揭示了神经代码生成系统中反馈机制的安全影响,为提高其安全性铺平了道路。
英文摘要:
Neural code generation systems have recently attracted increasing attention to improve developer productivity and speed up software development. Typically, these systems maintain a pre-trained neural model and make it available to general users as a service (e.g., through remote APIs) and incorporate a feedback mechanism to extensively collect and utilize the users' reaction to the generated code, i.e., user feedback. However, the security implications of such feedback have not yet been explored. With a systematic study of current feedback mechanisms, we find that feedback makes these systems vulnerable to feedback data injection (FDI) attacks. We discuss the methodology of FDI attacks and present a pre-attack profiling strategy to infer the attack constraints of a targeted system in the black-box setting. We demonstrate two proof-of-concept examples utilizing the FDI attack surface to implement prompt injection attacks and backdoor attacks on practical neural code generation systems. The attacker may stealthily manipulate a neural code generation system to generate code with vulnerabilities, attack payload, and malicious and spam messages. Our findings reveal the security implications of feedback mechanisms in neural code generation systems, paving the way for increasing their security.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。