arXiv 2601.032732026-03-19cs.CLcs.AIcs.HCcs.LG

一种多视角基准和评估模型用于评估安全性和对抗鲁棒性

A Multi-Perspective Benchmark and Moderation Model for Evaluating Safety and Adversarial Robustness

Department of Computer and Information Science, Fordham University（福德汉大学计算机与信息科学系）；College of Science and Engineering, Hamad Bin Khalifa University（哈马德·本·卡西姆大学科学与工程学院）；Department of Computer Science and Engineering, Qatar University（卡塔尔大学计算机科学与工程系）

Naseem Machlovi, Maryam Saleki, Ruhul Amin, Mohamed Rahouti, Shawqi Al-Maliki, Junaid Qadir, Mohamed M. Abdallah, Ala Al-Fuqaha

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出GuardEval多视角基准和GGuard模型，通过细粒度标签提升内容审核的准确性和对抗鲁棒性，实验显示GGuard在宏平均F1得分上优于现有模型。

摘要：

随着大语言模型（LLMs）深入日常生活，迫切需要更安全的审核系统，能够区分无害与有害请求并维持适当审查边界。现有LLMs在检测危险内容时表现良好，但在隐含冒犯、微妙性别和种族偏见及绕过提示等复杂案例中表现不佳，因这些问题具有主观性和上下文依赖性。此外，其对训练数据的依赖可能强化社会偏见，导致不一致和伦理问题。为此，我们引入GuardEval，一个统一的多视角基准数据集，包含106个细粒度类别，涵盖人类情感、攻击性语言、性别和种族偏见及更广泛的安全问题。我们还提出GGuard，一种基于量化低秩适应（QLoRA）微调的Gemma3-12B模型，训练于GuardEval以评估内容审核。评估显示，GGuard在宏平均F1得分为0.832，显著优于OpenAI Moderator（0.64）和Llama Guard（0.61）。我们表明，多视角、以人类为中心的安全基准对于缓解不一致的审核决策至关重要。GuardEval和GGuard共同证明，多样化和代表性数据在复杂和边缘案例中的安全性和对抗鲁棒性方面有实质性提升。

英文摘要：

As large language models (LLMs) become deeply embedded in daily life, the urgent need for safer moderation systems that distinguish between naive and harmful requests while upholding appropriate censorship boundaries has never been greater. While existing LLMs can detect dangerous or unsafe content, they often struggle with nuanced cases such as implicit offensiveness, subtle gender and racial biases, and jailbreak prompts, due to the subjective and context-dependent nature of these issues. Furthermore, their heavy reliance on training data can reinforce societal biases, resulting in inconsistent and ethically problematic outputs. To address these challenges, we introduce GuardEval, a unified multi-perspective benchmark dataset designed for both training and evaluation, containing 106 fine-grained categories spanning human emotions, offensive and hateful language, gender and racial bias, and broader safety concerns. We also present GemmaGuard (GGuard), a Quantized Low-Rank Adaptation (QLoRA), fine-tuned version of Gemma3-12B trained on GuardEval, to assess content moderation with fine-grained labels. Our evaluation shows that GGuard achieves a macro F1 score of 0.832, substantially outperforming leading moderation models, including OpenAI Moderator (0.64) and Llama Guard (0.61). We show that multi-perspective, human-centered safety benchmarks are critical for mitigating inconsistent moderation decisions. GuardEval and GGuard together demonstrate that diverse, representative data materially improve safety, and adversarial robustness on complex, borderline cases.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。