针对大型语言模型的对抗性演示攻击
Adversarial Demonstration Attacks on Large Language Models
AI总结:
本文提出advICL及可迁移版本Transferable-advICL,通过仅操纵演示而不改变输入来攻击大型语言模型的上下文学习,并证明演示数量增加会降低鲁棒性,且生成的对抗演示可攻击未见输入。
摘要:
随着ChatGPT和GPT-4等更强大的大型语言模型(LLMs)的出现,上下文学习(ICL)通过利用数据-标签对作为前置条件提示,在利用这些模型执行特定任务方面获得了显著 prominence。虽然加入演示可以极大地提升LLMs在各种任务上的性能,但它可能引入新的安全担忧:攻击者可以仅操纵演示而不改变输入来实施攻击。在本文中,我们从对抗性角度研究ICL的安全问题,重点关注演示的影响。我们提出了一种名为advICL的新型攻击方法,旨在仅操纵演示而不改变输入以误导模型。我们的结果表明,随着演示数量的增加,上下文学习的鲁棒性会降低。此外,我们还识别出演示的内在属性是它们可以(前置)与不同输入一起使用。因此,这引入了一种更实际的威胁模型,其中攻击者可以在不知道甚至不操纵测试输入示例的情况下攻击它。为实现这一点,我们提出了advICL的可迁移版本,名为Transferable-advICL。我们的实验表明,由Transferable-advICL生成的对抗性演示可以成功攻击未见过的测试输入示例。我们希望我们的研究揭示与ICL相关的关键安全风险,并强调对ICL鲁棒性进行广泛研究的必要性,特别是考虑到它在LLMs发展中的日益重要性。
英文摘要:
With the emergence of more powerful large language models (LLMs), such as ChatGPT and GPT-4, in-context learning (ICL) has gained significant prominence in leveraging these models for specific tasks by utilizing data-label pairs as precondition prompts. While incorporating demonstrations can greatly enhance the performance of LLMs across various tasks, it may introduce a new security concern: attackers can manipulate only the demonstrations without changing the input to perform an attack. In this paper, we investigate the security concern of ICL from an adversarial perspective, focusing on the impact of demonstrations. We propose a novel attack method named advICL, which aims to manipulate only the demonstration without changing the input to mislead the models. Our results demonstrate that as the number of demonstrations increases, the robustness of in-context learning would decrease. Additionally, we also identify the intrinsic property of the demonstrations is that they can be used (prepended) with different inputs. As a result, it introduces a more practical threat model in which an attacker can attack the test input example even without knowing and manipulating it. To achieve it, we propose the transferable version of advICL, named Transferable-advICL. Our experiment shows that the adversarial demonstration generated by Transferable-advICL can successfully attack the unseen test input examples. We hope that our study reveals the critical security risks associated with ICL and underscores the need for extensive research on the robustness of ICL, particularly given its increasing significance in the advancement of LLMs.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。