arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2202.12459cs.CL
收藏

APEACH:通过分析群体生成的仇恨语音评估数据集来攻击贬义表达

APEACH: Attacking Pejorative Expressions with Analysis on Crowd-Generated Hate Speech Evaluation Datasets

LG AI Research(LG人工智能研究院);Kakao Corp.(Kakao公司);Kakao Enterprise Corp.(Kakao企业公司);Soongsil University(崇实大学);Seoul National University(首尔大学)

Kichang Yang, Wonjun Jang, Won Ik Cho

摘要:

在仇恨言论检测中,开发跨各个领域的训练和评估数据集是关键问题。然而,主要的方法是抓取社交媒体文本,并雇佣人群工作者来注释数据。遵循这一惯例,往往将贬义词的范围限制在一个缺乏概括性的领域。在低数据语言上预训练语言模型时,有时训练语料和评估集之间的领域重叠会高估预测性能。为了缓解这些问题,在韩国,我们建议APEACH,要求未指定的用户生成仇恨言论的例子,然后最小的后标签。我们发现,APEACH可以收集有用的数据集,这些数据集对预训练语料库和评估集之间的词汇重叠不太敏感,从而正确地测量模型性能。

英文摘要:

In hate speech detection, developing training and evaluation datasets across various domains is the critical issue. Whereas, major approaches crawl social media texts and hire crowd-workers to annotate the data. Following this convention often restricts the scope of pejorative expressions to a single domain lacking generalization. Sometimes domain overlap between training corpus and evaluation set overestimate the prediction performance when pretraining language models on low-data language. To alleviate these problems in Korean, we propose APEACH that asks unspecified users to generate hate speech examples followed by minimal post-labeling. We find that APEACH can collect useful datasets that are less sensitive to the lexical overlaps between the pretraining corpus and the evaluation set, thereby properly measuring the model performance.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑