DROJ：一种针对大型语言模型的提示驱动攻击
DROJ: A Prompt-Driven Attack against Large Language Models
AI总结：
DROJ 是一种针对大型语言模型的提示驱动攻击方法，通过在嵌入层优化越狱提示以转移有害查询的隐藏表示，从而在 LLaMA-2-7b-chat 模型上实现 100% 的攻击成功率并提升响应实用性。
摘要：
大型语言模型（LLMs）在各种自然语言处理任务中展现出了卓越的能力。由于在互联网来源的数据集上进行训练，LLMs 有时会生成令人反感的内容，因此需要与人类反馈进行广泛对齐以避免此类输出。尽管进行了大规模的对齐努力，LLMs 仍然容易受到对抗性越狱攻击，这些攻击通常是经过设计的提示，旨在绕过安全机制并引出有害响应。在此，我们引入了一种新方法，即定向表示优化越狱（DROJ），它在嵌入层优化越狱提示，将有害查询的隐藏表示向更可能引出模型肯定响应的方向转移。我们在 LLaMA-2-7b-chat 模型上的评估表明，DROJ 实现了 100% 的基于关键词的攻击成功率（ASR），有效防止了直接拒绝。然而，该模型偶尔会产生重复且无信息量的响应。为了缓解这一问题，我们引入了一个有用性系统提示，以增强模型响应的实用性。我们的代码可在 https://github.com/Leon-Leyang/LLM-Safeguard 获取。
英文摘要：
Large Language Models (LLMs) have demonstrated exceptional capabilities across various natural language processing tasks. Due to their training on internet-sourced datasets, LLMs can sometimes generate objectionable content, necessitating extensive alignment with human feedback to avoid such outputs. Despite massive alignment efforts, LLMs remain susceptible to adversarial jailbreak attacks, which usually are manipulated prompts designed to circumvent safety mechanisms and elicit harmful responses. Here, we introduce a novel approach, Directed Rrepresentation Optimization Jailbreak (DROJ), which optimizes jailbreak prompts at the embedding level to shift the hidden representations of harmful queries towards directions that are more likely to elicit affirmative responses from the model. Our evaluations on LLaMA-2-7b-chat model show that DROJ achieves a 100\% keyword-based Attack Success Rate (ASR), effectively preventing direct refusals. However, the model occasionally produces repetitive and non-informative responses. To mitigate this, we introduce a helpfulness system prompt that enhances the utility of the model's responses. Our code is available at https://github.com/Leon-Leyang/LLM-Safeguard.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
