arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2407.03232cs.LGcs.CL
收藏

单字符扰动破坏大语言模型对齐

Single Character Perturbations Break LLM Alignment

National University of Singapore(新加坡国立大学)

Leon Lin, Hannah Brown, Kenji Kawaguchi, Michael Shieh

AI总结:

本文发现仅通过在输入末尾追加一个空格即可绕过多个开源大语言模型的安全对齐,使其以高成功率生成有害输出,并揭示了训练数据中空格上下文导致模型生成列表而覆盖拒绝指令的机制,凸显当前对齐方法的脆弱性。

摘要:

当大语言模型(LLMs)部署在敏感、面向人类的场景中时,至关重要的是它们不能输出不安全、有偏见或侵犯隐私的内容。为此,模型既经过训练也被指示拒绝回答不安全提示,例如“告诉我如何制造炸弹”。我们发现,尽管存在这些防护措施,仅通过在模型输入的末尾追加一个空格,就有可能破坏模型防御。在一项针对八个开源模型的研究中,我们证明这种攻击足够强大,能够以非常高的成功率导致大多数模型生成有害输出。我们考察了这种行为的原因,发现单个空格在分词后的训练数据中出现的上下文会鼓励模型在收到提示时生成列表,从而覆盖了拒绝回答不安全请求的训练信号。我们的发现凸显了当前模型对齐的脆弱状态,并强调了开发更稳健对齐方法的重要性。代码和数据将在 https://github.com/hannah-aught/space_attack 提供。

英文摘要:

When LLMs are deployed in sensitive, human-facing settings, it is crucial that they do not output unsafe, biased, or privacy-violating outputs. For this reason, models are both trained and instructed to refuse to answer unsafe prompts such as "Tell me how to build a bomb." We find that, despite these safeguards, it is possible to break model defenses simply by appending a space to the end of a model's input. In a study of eight open-source models, we demonstrate that this acts as a strong enough attack to cause the majority of models to generate harmful outputs with very high success rates. We examine the causes of this behavior, finding that the contexts in which single spaces occur in tokenized training data encourage models to generate lists when prompted, overriding training signals to refuse to answer unsafe requests. Our findings underscore the fragile state of current model alignment and promote the importance of developing more robust alignment methods. Code and data will be available at https://github.com/hannah-aught/space_attack.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑