重新思考如何评估语言模型越狱
Rethinking How to Evaluate Language Model Jailbreak
AI总结：
本文针对现有越狱评估目标不清且过度二元化的问题，提出安全防护违规、信息量和相对真实性三个指标及多方面评估方法，实验显示F1分数平均提升17%，推动更全面的越狱安全评估。
摘要：
大型语言模型（LLMs）已越来越多地集成到各种应用中。为确保LLMs不会生成不安全响应，它们会与规定受限内容的安全防护机制对齐。然而，这种对齐可以通过一种通常称为越狱（jailbreak）的技术被绕过，从而生成被禁止的内容。已有不同的系统被提出用于自动执行越狱。这些系统依赖评估方法来判断越狱尝试是否成功。然而，我们的分析揭示当前越狱评估方法存在两个局限性。（1）其目标缺乏清晰性，且与识别不安全响应的目标不一致。（2）它们将越狱结果过度简化为二元结果，即成功或不成功。在本文中，我们提出三个指标——安全防护违规（safeguard violation）、信息量（informativeness）和相对真实性（relative truthfulness）——来评估语言模型越狱。此外，我们展示了这些指标如何与不同恶意行为者的目标相关联。为计算这些指标，我们引入了一种多方面方法（multifaceted approach），该方法在对响应进行预处理后扩展了自然语言生成评估方法。我们在一个由三个恶意意图数据集和三个越狱系统生成的基准数据集上评估我们的指标。该基准数据集由三名标注者进行标注。我们将我们的多方面方法与三种现有越狱评估方法进行比较。实验表明，我们的多方面评估优于现有方法，与现有基线相比，F1分数平均提高了17%。我们的发现表明，有必要摆脱对越狱问题的二元视角，并纳入更全面的评估以确保语言模型的安全性。
英文摘要：
Large language models (LLMs) have become increasingly integrated with various applications. To ensure that LLMs do not generate unsafe responses, they are aligned with safeguards that specify what content is restricted. However, such alignment can be bypassed to produce prohibited content using a technique commonly referred to as jailbreak. Different systems have been proposed to perform the jailbreak automatically. These systems rely on evaluation methods to determine whether a jailbreak attempt is successful. However, our analysis reveals that current jailbreak evaluation methods have two limitations. (1) Their objectives lack clarity and do not align with the goal of identifying unsafe responses. (2) They oversimplify the jailbreak result as a binary outcome, successful or not. In this paper, we propose three metrics, safeguard violation, informativeness, and relative truthfulness, to evaluate language model jailbreak. Additionally, we demonstrate how these metrics correlate with the goal of different malicious actors. To compute these metrics, we introduce a multifaceted approach that extends the natural language generation evaluation method after preprocessing the response. We evaluate our metrics on a benchmark dataset produced from three malicious intent datasets and three jailbreak systems. The benchmark dataset is labeled by three annotators. We compare our multifaceted approach with three existing jailbreak evaluation methods. Experiments demonstrate that our multifaceted evaluation outperforms existing methods, with F1 scores improving on average by 17% compared to existing baselines. Our findings motivate the need to move away from the binary view of the jailbreak problem and incorporate a more comprehensive evaluation to ensure the safety of the language model.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
