Prompt Packer：通过隐藏攻击的组合指令欺骗大型语言模型
Prompt Packer: Deceiving LLMs through Compositional Instruction with Hidden Attacks
AI总结：
本文提出组合指令攻击（CIA），通过将有害指令伪装成无害任务来欺骗LLMs，在多个模型上实现高攻击成功率，揭示了LLMs的安全漏洞。
摘要：
近年来，具有强大通用能力的大型语言模型（LLMs）在经历对齐训练以确保生成内容符合用户意图和伦理的同时，被越来越多地集成到各种Web应用中。然而，它们在实际应用中仍然存在生成仇恨言论和犯罪活动等有害内容的风险。当前的方法主要依赖于检测、收集有害提示并针对其进行训练来防止此类风险。然而，这些方法通常侧重于具有单一意图的“表面”有害提示，忽略了在现实场景中容易引发有害内容的具有多种意图的组合攻击指令。在本文中，我们介绍了一种用于混淆有害指令的创新技术：组合指令攻击（CIA），它指通过组合和封装多个指令来进行攻击。CIA将有害提示隐藏在无害意图的指令中，使模型无法识别潜在的恶意意图。此外，我们实现了两种转换方法，称为T-CIA和W-CIA，以自动将有害指令伪装成对话或写作任务，使它们对LLMs看起来无害。我们使用两个安全评估数据集和两个有害提示数据集在GPT-4、ChatGPT和ChatGLM2上评估了CIA。它在安全评估数据集上实现了95%以上的攻击成功率，在有害提示数据集上对GPT-4实现了83%以上，对ChatGPT（基于gpt-3.5-turbo）和ChatGLM2-6B实现了91%以上的攻击成功率。我们的方法揭示了LLMs对这类隐藏潜在恶意意图的组合指令攻击的脆弱性，为LLM安全发展做出了重要贡献。警告：本文可能包含冒犯性或令人不安的内容！
英文摘要：
Recently, Large language models (LLMs) with powerful general capabilities have been increasingly integrated into various Web applications, while undergoing alignment training to ensure that the generated content aligns with user intent and ethics. Unfortunately, they remain the risk of generating harmful content like hate speech and criminal activities in practical applications. Current approaches primarily rely on detecting, collecting, and training against harmful prompts to prevent such risks. However, they typically focused on the "superficial" harmful prompts with a solitary intent, ignoring composite attack instructions with multiple intentions that can easily elicit harmful content in real-world scenarios. In this paper, we introduce an innovative technique for obfuscating harmful instructions: Compositional Instruction Attacks (CIA), which refers to attacking by combination and encapsulation of multiple instructions. CIA hides harmful prompts within instructions of harmless intentions, making it impossible for the model to identify underlying malicious intentions. Furthermore, we implement two transformation methods, known as T-CIA and W-CIA, to automatically disguise harmful instructions as talking or writing tasks, making them appear harmless to LLMs. We evaluated CIA on GPT-4, ChatGPT, and ChatGLM2 with two safety assessment datasets and two harmful prompt datasets. It achieves an attack success rate of 95%+ on safety assessment datasets, and 83%+ for GPT-4, 91%+ for ChatGPT (gpt-3.5-turbo backed) and ChatGLM2-6B on harmful prompt datasets. Our approach reveals the vulnerability of LLMs to such compositional instruction attacks that harbor underlying harmful intentions, contributing significantly to LLM security development. Warning: this paper may contain offensive or upsetting content!
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
