Latent Jailbreak：一个用于评估大语言模型文本安全性与输出鲁棒性的基准
Latent Jailbreak: A Benchmark for Evaluating Text Safety and Output Robustness of Large Language Models
AI总结：
该研究提出同时评估大语言模型安全性与鲁棒性的Latent Jailbreak基准，构建含嵌入恶意指令的潜在越狱提示数据集，通过多维度分析发现现有模型对不同指令动词存在响应优先级差异与越狱率区别。
摘要：
已有大量研究致力于确保大语言模型（LLMs）符合人类价值观并生成安全文本。然而，过度关注对特定主题的敏感性会损害模型遵循指令的鲁棒性，进而影响其完成任务的整体性能。此前针对LLMs越狱的基准测试主要聚焦于评估模型安全性，未考虑其鲁棒性。\n本文提出一个同时评估LLMs安全性与鲁棒性的基准，强调需采用平衡的评估思路。为全面研究文本安全性与输出鲁棒性，我们构建了一个潜在越狱提示数据集，每条提示均包含嵌入的恶意指令。具体而言，我们指示模型完成翻译等常规任务，但待翻译文本中包含恶意指令。为进一步分析安全性与鲁棒性，我们设计了一个分层标注框架。\n我们从显式常规指令的位置、词语替换（显式常规指令中的动词、恶意指令中的目标群体、显式常规指令的提示词）以及指令替换（不同的显式常规指令）三个维度，对LLMs的安全性与鲁棒性展开系统分析。结果表明，当前的LLMs不仅会优先响应特定指令动词，且对显式常规指令中不同的指令动词表现出不同的越狱率。代码与数据可在https://github.com/qiuhuachuan/latent-jailbreak获取。
英文摘要：
Considerable research efforts have been devoted to ensuring that large language models (LLMs) align with human values and generate safe text. However, an excessive focus on sensitivity to certain topics can compromise the model's robustness in following instructions, thereby impacting its overall performance in completing tasks. Previous benchmarks for jailbreaking LLMs have primarily focused on evaluating the safety of the models without considering their robustness. In this paper, we propose a benchmark that assesses both the safety and robustness of LLMs, emphasizing the need for a balanced approach. To comprehensively study text safety and output robustness, we introduce a latent jailbreak prompt dataset, each involving malicious instruction embedding. Specifically, we instruct the model to complete a regular task, such as translation, with the text to be translated containing malicious instructions. To further analyze safety and robustness, we design a hierarchical annotation framework. We present a systematic analysis of the safety and robustness of LLMs regarding the position of explicit normal instructions, word replacements (verbs in explicit normal instructions, target groups in malicious instructions, cue words for explicit normal instructions), and instruction replacements (different explicit normal instructions). Our results demonstrate that current LLMs not only prioritize certain instruction verbs but also exhibit varying jailbreak rates for different instruction verbs in explicit normal instructions. Code and data are available at https://github.com/qiuhuachuan/latent-jailbreak.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
