SG-Bench：评估大语言模型在多样化任务与提示类型下的安全泛化能力
SG-Bench: Evaluating LLM Safety Generalization Across Diverse Tasks and Prompt Types
AI总结：
针对现有LLM安全基准评估范式单一、未考虑提示技术影响的问题，研究提出融合两类评估任务、覆盖提示工程与越狱场景的SG-Bench基准，测评发现多数LLM安全泛化能力不足。
摘要：
确保大语言模型（LLM）应用的安全性是发展可信人工智能的核心要务。当前LLM安全基准存在两大局限：其一，仅聚焦判别式或生成式单一评估范式，忽略了二者的内在关联；其二，依赖标准化输入，未考虑系统提示、少样本示例、思维链提示等广泛应用的提示技术的影响。为解决上述问题，我们构建了SG-Bench——一款用于评估LLM安全能力在不同任务与提示类型间泛化表现的新型基准。该基准融合了生成式与判别式两类评估任务，还包含扩展数据集，用于检验提示工程与越狱攻击对LLM安全性的影响。我们用该基准测评了3款先进的闭源LLM与10款开源LLM，结果显示多数LLM在判别式任务上的表现弱于生成式任务，且对提示改动高度敏感，表明其安全对齐的泛化能力较差。我们还从定量与定性角度解释了这些发现，为后续研究提供参考。
英文摘要：
Ensuring the safety of large language model (LLM) applications is essential for developing trustworthy artificial intelligence. Current LLM safety benchmarks have two limitations. First, they focus solely on either discriminative or generative evaluation paradigms while ignoring their interconnection. Second, they rely on standardized inputs, overlooking the effects of widespread prompting techniques, such as system prompts, few-shot demonstrations, and chain-of-thought prompting. To overcome these issues, we developed SG-Bench, a novel benchmark to assess the generalization of LLM safety across various tasks and prompt types. This benchmark integrates both generative and discriminative evaluation tasks and includes extended data to examine the impact of prompt engineering and jailbreak on LLM safety. Our assessment of 3 advanced proprietary LLMs and 10 open-source LLMs with the benchmark reveals that most LLMs perform worse on discriminative tasks than generative ones, and are highly susceptible to prompts, indicating poor generalization in safety alignment. We also explain these findings quantitatively and qualitatively to provide insights for future research.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
