重审仇恨言论基准：从数据构建到系统部署
Revisiting Hate Speech Benchmarks: From Data Curation to System Deployment
AI总结：
针对现有仇恨言论基准数据集难以覆盖现实多样性的问题，本文提出以中性内容为种子的多语言大规模数据集GOTHate，设计融合内生信号的HEN-mBERT模型，性能优于基线，并与企业合作开发半自动化检测流程。
摘要：
社交媒体上充斥着仇恨内容，其中很多往往隐藏在语言和话题的多样性之下。用于仇恨言论检测的基准数据集并未考虑到这种偏离性，因为它们主要是使用仇恨词典构建的。然而，在以中性内容为种子筛选出的恶意内容中，捕捉仇恨信号变得极具挑战性。因此，设计能够模拟现实世界中仇恨内容多样性的模型和数据集值得进一步研究。\n为此，我们提出了GOTHate——一个来自Twitter、包含约5.1万条帖子的大规模代码混合众包仇恨言论检测数据集。GOTHate以中性内容为种子构建，涵盖不同的语言和话题。我们将GOTHate与现有仇恨言论数据集进行了详细对比，凸显了其新颖性。我们用10个最新基线模型在该数据集上进行了基准测试。大量实证和基准实验表明，在纯文本设置下GOTHate的分类难度很高。因此，我们探究了加入内生信号如何提升仇恨言论检测任务的效果。我们为GOTHate补充了用户的时间线信息和自我网络，使整体数据源更接近理解仇恨内容的现实场景。我们提出的解决方案HEN-mBERT是一个模块化的多语言混合专家模型，它利用来自历史、拓扑结构和样例的潜在内生信号丰富了语言子空间。HEN-mBERT在整体宏F1值和仇恨类别F1值上分别超越最优基线2.5%和5%。受实验启发，我们与Wipro AI合作，正在开发一套半自动化的仇恨内容检测流程，作为其应对网络危害使命的一部分。
英文摘要：
Social media is awash with hateful content, much of which is often veiled with linguistic and topical diversity. The benchmark datasets used for hate speech detection do not account for such divagation as they are predominantly compiled using hate lexicons. However, capturing hate signals becomes challenging in neutrally-seeded malicious content. Thus, designing models and datasets that mimic the real-world variability of hate warrants further investigation. To this end, we present GOTHate, a large-scale code-mixed crowdsourced dataset of around 51k posts for hate speech detection from Twitter. GOTHate is neutrally seeded, encompassing different languages and topics. We conduct detailed comparisons of GOTHate with the existing hate speech datasets, highlighting its novelty. We benchmark it with 10 recent baselines. Our extensive empirical and benchmarking experiments suggest that GOTHate is hard to classify in a text-only setup. Thus, we investigate how adding endogenous signals enhances the hate speech detection task. We augment GOTHate with the user's timeline information and ego network, bringing the overall data source closer to the real-world setup for understanding hateful content. Our proposed solution HEN-mBERT is a modular, multilingual, mixture-of-experts model that enriches the linguistic subspace with latent endogenous signals from history, topology, and exemplars. HEN-mBERT transcends the best baseline by 2.5% and 5% in overall macro-F1 and hate class F1, respectively. Inspired by our experiments, in partnership with Wipro AI, we are developing a semi-automated pipeline to detect hateful content as a part of their mission to tackle online harm.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
