arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1803.05495cs.CL
收藏

区分亵渎和仇恨言论的挑战

Challenges in Discriminating Profanity from Hate Speech

Harvard Medical School(哈佛医学院);University of Wolverhampton(沃尔夫汉普顿大学)

Shervin Malmasi, Marcos Zampieri

摘要:

在这项研究中,我们将探讨如何区分社交媒体中的一般亵渎和仇恨言论,这一点尚未得到广泛考虑。使用专门为此任务注释的新数据集,我们采用监督分类以及一组特征,包括n-gram,skip-gram和基于聚类的单词表示。我们应用基于单分类器以及更高级的集成分类器和堆叠泛化的方法,实现了这个3类分类任务80%准确率的最佳结果。对结果的分析表明,区分仇恨言论和亵渎并不是一项简单的任务,这可能需要能够更深入地理解文本的特征,而这些特征并不总是能够通过表面n-gram来实现。由于注释者主观判断的差异,注释数据中黄金标签的可变性也是一个问题。讨论了今后工作的其他方向。

英文摘要:

In this study we approach the problem of distinguishing general profanity from hate speech in social media, something which has not been widely considered. Using a new dataset annotated specifically for this task, we employ supervised classification along with a set of features that includes n-grams, skip-grams and clustering-based word representations. We apply approaches based on single classifiers as well as more advanced ensemble classifiers and stacked generalization, achieving the best result of 80% accuracy for this 3-class classification task. Analysis of the results reveals that discriminating hate speech and profanity is not a simple task, which may require features that capture a deeper understanding of the text not always possible with surface n-grams. The variability of gold labels in the annotated data, due to differences in the subjective adjudications of the annotators, is also an issue. Other directions for future work are discussed.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑