arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2603.01724cs.AI
收藏

GMP:在同时发生违规和动态规则下的内容审查基准

GMP: A Benchmark for Content Moderation under Co-occurring Violations and Dynamic Rules

Beijing University of Posts and Telecommunications(北京邮电大学);The University of Hong Kong(香港大学)

Houde Dong, Yifei She, Kai Ye, Liangcai Su, Chenxiong Qian, Jie Hao

AI总结:

GMP提出一个基准,用于评估AI在同时发生违规和动态规则下的内容审查能力。

摘要:

在线内容审查对于维护健康的数字环境至关重要,且依赖AI进行此任务的力度持续增长。考虑一个用户评论使用国家刻板印象来辱骂政治家的例子。这个例子展示了现实场景中的两个关键挑战:(1)同时发生的违规,即一条帖子违反多项政策(例如偏见和人身攻击);(2)动态规则,即确定违规取决于平台特定的指南,这些指南在不同上下文中不断变化。同时发生的危害与不断变化的规则的交汇突显了当前AI系统的核心限制:尽管大型语言模型(LLMs)擅长遵循固定指南,但当政策不稳定或依赖上下文时,其判断能力会下降。在实践中,这些不足会导致不一致的审查:要么错误地限制合法表达,要么让有害内容留在网上。这提出了一个关键的评估问题:在涉及同时发生的违规和动态变化的规则的现实场景中,现有静态基准的高性能是否真正保证了AI判断的稳健泛化?

英文摘要:

Online content moderation is essential for maintaining a healthy digital environment, and reliance on AI for this task continues to grow. Consider a user comment using national stereotypes to insult a politician. This example illustrates two critical challenges in real-world scenarios: (1) Co-occurring Violations, where a single post violates multiple policies (e.g., prejudice and personal attacks); (2) Dynamic rules of moderation, where determination of a violation depends on platform-specific guidelines that evolve across contexts . The intersection of co-occurring harms and dynamically changing rules highlights a core limitation of current AI systems: although large language models (LLMs) are adept at following fixed guidelines, their judgment capabilities degrade when policies are unstable or context-dependent . In practice, such shortcomings lead to inconsistent moderation: either erroneously restricting legitimate expression or allowing harmful content to remain online . This raises a critical question for evaluation: Does high performance on existing static benchmarks truly guarantee robust generalization of AI judgment to real-world scenarios involving co-occurring violations and dynamically changing rules?

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑