诱骗 LLMs 违抗指令:越狱的形式化、分析与检测
Tricking LLMs into Disobedience: Formalizing, Analyzing, and Detecting Jailbreaks
AI总结:
针对非专业用户通过操纵提示词越狱 LLMs 导致的安全与违规问题,本文提出越狱的形式化与分类体系,调研现有方法在多模型上的有效性,探讨检测挑战,并发布了 3700 个越狱提示词的模型输出数据集。
摘要:
近期对商业大型语言模型(LLMs)的探索表明,非专业用户仅需操纵提示词即可越狱 LLMs;导致输出行为退化、隐私与安全泄露、攻击性输出以及违反内容监管政策。目前针对这些攻击及其缓解措施进行形式化与分析的研究有限。我们通过提出已知(及潜在)越狱的形式化与分类体系来弥补这一空白。我们调研了现有的越狱方法及其在开源与商业 LLMs(如基于 GPT 的模型、OPT、BLOOM 和 FLAN-T5-XXL)上的有效性。我们进一步讨论了越狱检测在抵御已知攻击方面的有效性挑战。为进一步分析,我们发布了一个包含 4 项任务下 3700 个越狱提示词的模型输出数据集。
英文摘要:
Recent explorations with commercial Large Language Models (LLMs) have shown that non-expert users can jailbreak LLMs by simply manipulating their prompts; resulting in degenerate output behavior, privacy and security breaches, offensive outputs, and violations of content regulator policies. Limited studies have been conducted to formalize and analyze these attacks and their mitigations. We bridge this gap by proposing a formalism and a taxonomy of known (and possible) jailbreaks. We survey existing jailbreak methods and their effectiveness on open-source and commercial LLMs (such as GPT-based models, OPT, BLOOM, and FLAN-T5-XXL). We further discuss the challenges of jailbreak detection in terms of their effectiveness against known attacks. For further analysis, we release a dataset of model outputs across 3700 jailbreak prompts over 4 tasks.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。