HateModerate：针对内容审核策略测试仇恨言论检测器
HateModerate: Testing Hate Speech Detectors against Content Moderation Policies
AI总结：
本研究创建HateModerate数据集，用于测试仇恨言论检测器是否符合平台内容政策，并发现现有模型存在重大失败，通过数据增强显著提升了模型的政策符合度。
摘要：
为保护用户免受大量仇恨内容的侵害，已有研究致力于自动化仇恨言论检测。尽管已有这些努力，仍有一个问题悬而未决：自动化仇恨言论检测器是否符合社交媒体平台的内容政策？平台的内容政策是社交媒体平台审核内容的一份清单。由于内容审核规则通常是独特定义的，现有的仇恨言论数据集无法直接回答这一问题。本研究旨在通过创建HateModerate来回答该问题，这是一个用于测试自动化内容审核器行为是否符合内容政策的数据集。首先，我们让28名标注者和GPT参与一个六步标注过程，最终生成了与Facebook的41条仇恨言论政策分别匹配的仇恨性和非仇恨性测试套件列表。其次，我们测试了最先进的仇恨言论检测器在HateModerate上的表现，揭示了这些模型在符合政策方面存在的重大失败。第三，利用HateModerate，我们扩充了HuggingFace上一个下载量最高的仇恨检测器的训练数据。我们观察到模型在符合内容政策方面有显著提升，同时在原始测试数据上保持了可比的分数。我们的数据集和代码可在附件中找到。
英文摘要：
To protect users from massive hateful content, existing works studied automated hate speech detection. Despite the existing efforts, one question remains: do automated hate speech detectors conform to social media content policies? A platform's content policies are a checklist of content moderated by the social media platform. Because content moderation rules are often uniquely defined, existing hate speech datasets cannot directly answer this question. This work seeks to answer this question by creating HateModerate, a dataset for testing the behaviors of automated content moderators against content policies. First, we engage 28 annotators and GPT in a six-step annotation process, resulting in a list of hateful and non-hateful test suites matching each of Facebook's 41 hate speech policies. Second, we test the performance of state-of-the-art hate speech detectors against HateModerate, revealing substantial failures these models have in their conformity to the policies. Third, using HateModerate, we augment the training data of a top-downloaded hate detector on HuggingFace. We observe significant improvement in the models' conformity to content policies while having comparable scores on the original test data. Our dataset and code can be found in the attachment.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
