发表机构
Georgia Institute of Technology; Snowflake（佐治亚理工学院; 雪花公司）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本文提出EviLLM攻击，通过第三方对手破坏AI代码生成流水线，利用受损账户或浏览器访问LLM，从13个CWE类别注入漏洞，用户研究显示多数参与者未能察觉注入的漏洞。
AI 中文摘要
大型语言模型（LLM）的进步使得从自然语言规范进行AI驱动的代码生成成为可能，这为向软件中注入漏洞引入了新的攻击面。先前的工作仅在良性环境中研究过此问题，即漏洞是无意引入的，或者在不寻常的威胁模型下，其中LLM本身是恶意的（后门攻击）或用户是攻击者（越狱）。在本文中，我们研究了一个更现实的威胁模型：一个第三方对手，其能力与现有网络犯罪分子相当，破坏AI代码生成流水线以故意引入漏洞。我们称之为EviLLM攻击。我们实现了EviLLM的两个实例，每个实例仅需要通过受损账户或浏览器访问底层LLM，并且可以从13个CWE类别注入漏洞。正如我们在可行性研究中所展示的，这两种攻击向量已经被用于实施许多现有的网络攻击。我们的用户研究表明，8名参与者中有7名和13名参与者中有10名没有注意到由EviLLM的两个实例注入的漏洞，而21名参与者中有13名“很少”或“从不”考虑像EviLLM这样的攻击风险。
英文摘要
Advances in large language models (LLMs) have enabled AI-driven code generation from natural language specifications, introducing new attack surfaces for injecting vulnerabilities into software. Prior work has studied this problem only in benign settings where vulnerabilities are introduced inadvertently, or under unconventional threat models where the LLM itself is malicious (backdooring) or the user is the attacker (jailbreaking). In this paper, we study a more realistic threat model: a third-party adversary, with capabilities comparable to existing cybercriminals, compromises the AI code generation pipeline to deliberately introduce vulnerabilities. We call this the EviLLM attack. We have implemented two instances of EviLLM, each of which only requires the underlying LLM to be accessed through a compromised account or browser, and can inject vulnerabilities from 13 CWE classes. As we show in our feasibility study, both attack vectors are already used to implement many existing cyberattacks. Our user study shows that 7 out of 8 and 10 out of 13 participants did not notice the vulnerabilities injected by the two instances of EviLLM, and 13 out of 21 participants "rarely" or "never" considered the risk of an attack like EviLLM.
