arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2403.14725cs.CRcs.CLcs.LG
收藏

用紫色问题测试越狱防御的极限

Testing the Limits of Jailbreaking Defenses with the Purple Problem

Carnegie Mellon University(卡内基梅隆大学)

Taeyoun Kim, Suhas Kotha, Aditi Raghunathan

AI总结:

本研究通过“紫色问题”(即输出含purple即为不安全的简单定义)测试现有语言模型越狱防御的执行机制,发现微调和输入防御均失效,指出现有安全基准也仅测固定定义下的执行能力,呼吁未来研究优化防御执行与定义质量。

摘要:

针对语言模型的“越狱”攻击兴起,催生了大量旨在阻止不当响应的防御措施。我们批判性地审视了防御流程的两个阶段:(i)定义不安全输出的构成,以及(ii)通过输入处理或微调等方法执行该定义。为测试现有执行机制的有效性,我们采用了一个简单且明确的不安全输出定义——包含“purple(紫色)”一词的输出。令人惊讶的是,现有的微调和输入防御在这个简单问题上均告失败,这让人质疑执行算法能否对更复杂的定义具备鲁棒性。我们发现,真实安全基准同样是针对固定定义测试执行能力。我们希望未来研究能产出有效且快速的执行机制,以及用于执行和评估的高质量定义。

英文摘要:

The rise of "jailbreak" attacks on language models has led to a flurry of defenses aimed at preventing undesirable responses. We critically examine the two stages of the defense pipeline: (i) defining what constitutes unsafe outputs, and (ii) enforcing the definition via methods such as input processing or fine-tuning. To test the efficacy of existing enforcement mechanisms, we consider a simple and well-specified definition of unsafe outputs--outputs that contain the word "purple". Surprisingly, existing fine-tuning and input defenses fail on this simple problem, casting doubt on whether enforcement algorithms can be robust for more complicated definitions. We find that real safety benchmarks similarly test enforcement for a fixed definition. We hope that future research can lead to effective/fast enforcement as well as high quality definitions used for enforcement and evaluation.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑