arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2410.02841cs.CRcs.SE
收藏

针对代码智能中上下文学习的演示攻击

Demonstration Attack against In-Context Learning for Code Intelligence

Yifei Ge, Weisong Sun, Yihang Lou, Chunrong Fang, Yiran Zhang, Yiming Li, Xiaofang Zhang, Yang Liu, Zhihong Zhao, Zhenyu Chen

AI总结:

本文研究代码智能中上下文学习(ICL)的安全风险,提出名为DICE的两阶段攻击方法,通过恶意演示构造可迁移的恶意ICL内容以误导LLMs输出,揭示了保护ICL机制的重要性。

摘要:

大语言模型(LLMs)的最新进展通过提高编程生产力并缓解软件开发者面临的挑战,彻底变革了代码智能领域。为了进一步提升LLMs在特定代码智能任务上的表现并降低训练成本,研究人员揭示了LLMs的一项新能力:上下文学习(ICL)。ICL允许LLMs在特定上下文中从少量演示中学习,无需更新参数即可取得令人印象深刻的结果。然而,ICL的兴起在代码智能领域引入了新的安全漏洞。在本文中,我们探索了一种基于ICL范式的全新安全场景,其中攻击者充当第三方ICL代理机构,向用户提供恶意ICL内容,以在代码智能任务中误导LLMs的输出。我们的研究证明了该场景的可行性与风险,揭示了攻击者如何利用恶意演示来构造恶意ICL内容,并诱导LLMs产生错误输出,从而对系统安全构成重大威胁。我们提出了一种名为DICE的新方法来构造恶意ICL内容,该方法由两个阶段组成:演示选择与恶意ICL构造,基于用户查询构造有针对性的恶意ICL内容,并且可跨不同查询输入迁移。最终,我们的发现强调了保护ICL机制以保护代码智能系统免受对抗性操纵的至关重要性。

英文摘要:

Recent advancements in large language models (LLMs) have revolutionized code intelligence by improving programming productivity and alleviating challenges faced by software developers. To further improve the performance of LLMs on specific code intelligence tasks and reduce training costs, researchers reveal a new capability of LLMs: in-context learning (ICL). ICL allows LLMs to learn from a few demonstrations within a specific context, achieving impressive results without parameter updating. However, the rise of ICL introduces new security vulnerabilities in the code intelligence field. In this paper, we explore a novel security scenario based on the ICL paradigm, where attackers act as third-party ICL agencies and provide users with bad ICL content to mislead LLMs outputs in code intelligence tasks. Our study demonstrates the feasibility and risks of such a scenario, revealing how attackers can leverage malicious demonstrations to construct bad ICL content and induce LLMs to produce incorrect outputs, posing significant threats to system security. We propose a novel method to construct bad ICL content called DICE, which is composed of two stages: Demonstration Selection and Bad ICL Construction, constructing targeted bad ICL content based on the user query and transferable across different query inputs. Ultimately, our findings emphasize the critical importance of securing ICL mechanisms to protect code intelligence systems from adversarial manipulation.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑