FlexGuard: 严格度自适应的连续风险评分用于LLM内容审核
FlexGuard: Continuous Risk Scoring for Strictness-Adaptive LLM Content Moderation
AI总结:
本文提出FlexGuard,一种基于LLM的连续风险评分系统,通过风险对齐优化提升审核准确性和在不同严格度下的鲁棒性。
摘要:
确保LLM生成内容的安全性对实际部署至关重要。现有防护模型将审核视为固定二元分类任务,隐含假设有害性定义固定。实际中,执行严格度随平台变化且随时间演变,使二元审核器在需求变化时变得脆弱。我们首先引入FlexBench,一个严格度自适应的LLM审核基准,支持多严格度的受控评估。实验揭示现有审核器在不同严格度下存在显著不一致:在一种制度下表现良好的模型在其他制度下会显著退化,限制其实用性。为此,我们提出FlexGuard,一种基于LLM的审核器,输出校准的连续风险评分反映风险严重性,并通过阈值设置支持特定严格度决策。我们通过风险对齐优化训练FlexGuard以提高评分严重性一致性,并提供实用的阈值选择策略以适应部署目标严格度。在FlexBench和公共基准上的实验表明,FlexGuard在不同严格度下实现了更高的审核准确性和显著改进的鲁棒性。我们发布源代码和数据以支持可重复性。
英文摘要:
Ensuring the safety of LLM-generated content is essential for real-world deployment. Most existing guardrail models formulate moderation as a fixed binary classification task, implicitly assuming a fixed definition of harmfulness. In practice, enforcement strictness - how conservatively harmfulness is defined and enforced - varies across platforms and evolves over time, making binary moderators brittle under shifting requirements. We first introduce FlexBench, a strictness-adaptive LLM moderation benchmark that enables controlled evaluation under multiple strictness regimes. Experiments on FlexBench reveal substantial cross-strictness inconsistency in existing moderators: models that perform well under one regime can degrade substantially under others, limiting their practical usability. To address this, we propose FlexGuard, an LLM-based moderator that outputs a calibrated continuous risk score reflecting risk severity and supports strictness-specific decisions via thresholding. We train FlexGuard via risk-alignment optimization to improve score-severity consistency and provide practical threshold selection strategies to adapt to target strictness at deployment. Experiments on FlexBench and public benchmarks demonstrate that FlexGuard achieves higher moderation accuracy and substantially improved robustness under varying strictness. We release the source code and data to support reproducibility.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。