arXiv 2406.145982025-03-01cs.AI

SORRY-Bench：系统评估大型语言模型安全拒绝能力

SORRY-Bench: Systematically Evaluating Large Language Model Safety Refusal

Princeton University（普林斯顿大学）；Virginia Tech（弗吉尼亚理工大学）；Stanford University（斯坦福大学）；UC Berkeley（加州大学伯克利分校）；University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；University of Chicago（芝加哥大学）

Tinghao Xie, Xiangyu Qi, Yi Zeng, Yangsibo Huang, Udari Madhushani Sehwag, Kaixuan Huang, Luxi He, Boyi Wei, Dacheng Li, Ying Sheng, Ruoxi Jia, Bo Li, Kai Li, D… 展开作者

Tinghao Xie, Xiangyu Qi, Yi Zeng, Yangsibo Huang, Udari Madhushani Sehwag, Kaixuan Huang, Luxi He, Boyi Wei, Dacheng Li, Ying Sheng, Ruoxi Jia, Bo Li, Kai Li, Danqi Chen, Peter Henderson, Prateek Mittal

备注：: Paper accepted to ICLR 2025

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出SORRY-Bench基准，通过细粒度分类法、语言增强和高效自动评估器，系统评估并分析了50多个LLMs的安全拒绝行为，解决了现有评估方法在主题覆盖、语言特征和计算成本上的局限。

摘要：

评估已对齐的大型语言模型（LLMs）识别并拒绝不安全用户请求的能力，对于安全且符合政策的部署至关重要。然而，现有的评估工作存在三个局限性，我们通过提出的基准SORRY-Bench来解决这些问题。首先，现有方法通常使用粗粒度的不安全主题分类法，且过度代表了某些细粒度主题。例如，在我们评估的十个现有数据集中，对自残指令拒绝的测试代表性比欺诈活动测试低3倍以上。SORRY-Bench通过使用包含44个潜在不安全主题的细粒度分类法，以及通过人在环方法编制的440个类别平衡的不安全指令来改进这一点。其次，提示的语言特征和格式常被忽视，如不同语言、方言等，这些在许多评估中仅被隐式考虑。我们为SORRY-Bench补充了20种多样的语言增强，以系统地检查这些影响。第三，现有评估依赖大型LLMs（如GPT-4）进行评估，计算成本可能很高。我们研究了创建快速、准确的自动化安全评估器的设计选择。通过收集7K+人工标注并对多种LLM作为评判者的设计进行元评估，我们表明微调的7B LLMs能以较低计算成本达到与GPT-4规模LLMs相当的准确率。综合这些，我们在SORRY-Bench上评估了50多个专有和开放权重的LLMs，分析其独特的安全拒绝行为。我们希望这一努力能以平衡、细粒度且高效的方式，为系统评估LLMs安全拒绝能力提供基础。基准演示、数据、代码和模型可通过https://sorry-bench.github.io获取。

英文摘要：

Evaluating aligned large language models' (LLMs) ability to recognize and reject unsafe user requests is crucial for safe, policy-compliant deployments. Existing evaluation efforts, however, face three limitations that we address with SORRY-Bench, our proposed benchmark. First, existing methods often use coarse-grained taxonomies of unsafe topics, and are over-representing some fine-grained topics. For example, among the ten existing datasets that we evaluated, tests for refusals of self-harm instructions are over 3x less represented than tests for fraudulent activities. SORRY-Bench improves on this by using a fine-grained taxonomy of 44 potentially unsafe topics, and 440 class-balanced unsafe instructions, compiled through human-in-the-loop methods. Second, linguistic characteristics and formatting of prompts are often overlooked, like different languages, dialects, and more -- which are only implicitly considered in many evaluations. We supplement SORRY-Bench with 20 diverse linguistic augmentations to systematically examine these effects. Third, existing evaluations rely on large LLMs (e.g., GPT-4) for evaluation, which can be computationally expensive. We investigate design choices for creating a fast, accurate automated safety evaluator. By collecting 7K+ human annotations and conducting a meta-evaluation of diverse LLM-as-a-judge designs, we show that fine-tuned 7B LLMs can achieve accuracy comparable to GPT-4 scale LLMs, with lower computational cost. Putting these together, we evaluate over 50 proprietary and open-weight LLMs on SORRY-Bench, analyzing their distinctive safety refusal behaviors. We hope our effort provides a building block for systematic evaluations of LLMs' safety refusal capabilities, in a balanced, granular, and efficient manner. Benchmark demo, data, code, and models are available through https://sorry-bench.github.io.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。