APEACH:通过分析群体生成的仇恨语音评估数据集来攻击贬义表达
APEACH: Attacking Pejorative Expressions with Analysis on Crowd-Generated Hate Speech Evaluation Datasets
摘要:
在仇恨言论检测中,开发跨各个领域的训练和评估数据集是关键问题。然而,主要的方法是抓取社交媒体文本,并雇佣人群工作者来注释数据。遵循这一惯例,往往将贬义词的范围限制在一个缺乏概括性的领域。在低数据语言上预训练语言模型时,有时训练语料和评估集之间的领域重叠会高估预测性能。为了缓解这些问题,在韩国,我们建议APEACH,要求未指定的用户生成仇恨言论的例子,然后最小的后标签。我们发现,APEACH可以收集有用的数据集,这些数据集对预训练语料库和评估集之间的词汇重叠不太敏感,从而正确地测量模型性能。
英文摘要:
In hate speech detection, developing training and evaluation datasets across various domains is the critical issue. Whereas, major approaches crawl social media texts and hire crowd-workers to annotate the data. Following this convention often restricts the scope of pejorative expressions to a single domain lacking generalization. Sometimes domain overlap between training corpus and evaluation set overestimate the prediction performance when pretraining language models on low-data language. To alleviate these problems in Korean, we propose APEACH that asks unspecified users to generate hate speech examples followed by minimal post-labeling. We find that APEACH can collect useful datasets that are less sensitive to the lexical overlaps between the pretraining corpus and the evaluation set, thereby properly measuring the model performance.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。