arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2311.13445cs.LGcs.CR
收藏

大型语言模型在编程任务上的迁移攻击与防御

Transfer Attacks and Defenses for Large Language Models on Coding Tasks

Carnegie Mellon University(卡内基梅隆大学);Center for AI Safety(人工智能安全中心)

Chi Zhang, Zifan Wang, Ravi Mangal, Matt Fredrikson, Limin Jia, Corina Pasareanu

AI总结:

本文研究了针对大型语言模型编程任务的对抗扰动迁移攻击,并提出基于提示的防御方法,通过添加对抗样本示例和逆转指令提升模型鲁棒性。

摘要:

现代大型语言模型(LLMs),如ChatGPT,在编写代码和代码推理等编程任务上展现了令人印象深刻的能力。它们改进了之前的代码神经网络模型,如code2seq或seq2seq,这些模型在执行代码摘要和识别代码漏洞等任务时已经展示了有竞争力的结果。然而,这些先前的代码模型被证明容易受到对抗样本的攻击,即不改变程序语义的微小语法扰动,例如通过错误条件包含“死代码”或添加无关紧要的打印语句,旨在“愚弄”模型。LLMs也可能容易受到相同的对抗扰动的影响,但迄今为止缺乏对此问题的详细研究。在本文中,我们旨在研究对抗扰动对LLMs编程任务的影响。特别是,我们研究了通过对较小代码模型进行白盒攻击生成的对抗样本向LLMs的迁移性。此外,为了在不产生重训练成本的情况下使LLMs对这种对手更加鲁棒,我们提出了基于提示的防御方法,该方法涉及修改提示以包含额外信息,例如对抗扰动代码的示例和用于逆转对抗扰动的明确指令。我们的实验表明,使用较小代码模型获得的对抗样本确实具有迁移性,削弱了LLMs的性能。所提出的防御方法在提高模型的弹性方面显示出前景,为代码相关应用中更鲁棒的LLMs防御解决方案铺平了道路。

英文摘要:

Modern large language models (LLMs), such as ChatGPT, have demonstrated impressive capabilities for coding tasks including writing and reasoning about code. They improve upon previous neural network models of code, such as code2seq or seq2seq, that already demonstrated competitive results when performing tasks such as code summarization and identifying code vulnerabilities. However, these previous code models were shown vulnerable to adversarial examples, i.e. small syntactic perturbations that do not change the program's semantics, such as the inclusion of "dead code" through false conditions or the addition of inconsequential print statements, designed to "fool" the models. LLMs can also be vulnerable to the same adversarial perturbations but a detailed study on this concern has been lacking so far. In this paper we aim to investigate the effect of adversarial perturbations on coding tasks with LLMs. In particular, we study the transferability of adversarial examples, generated through white-box attacks on smaller code models, to LLMs. Furthermore, to make the LLMs more robust against such adversaries without incurring the cost of retraining, we propose prompt-based defenses that involve modifying the prompt to include additional information such as examples of adversarially perturbed code and explicit instructions for reversing adversarial perturbations. Our experiments show that adversarial examples obtained with a smaller code model are indeed transferable, weakening the LLMs' performance. The proposed defenses show promise in improving the model's resilience, paving the way to more robust defensive solutions for LLMs in code-related applications.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑