arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2207.04003cs.CL
收藏

时不我待:语言变化对自动评论审核的影响

No Time Like the Present: Effects of Language Change on Automated Comment Moderation

University of Wisconsin–Madison(威斯康星大学麦迪逊分校);European Research Center for Information Systems(欧洲信息系统研究中心)

Lennart Justen, Kilian Müller, Marco Niemann, Jörg Becker

摘要:

网络仇恨的传播已经成为报纸评论部分的一个重要问题。因此,人们越来越有兴趣使用机器学习和自然语言处理进行(半)自动化的滥用语言检测,以避免手动评论审核成本或不得不完全关闭评论部分。然而,许多过去的工作对滥用语言检测假设,分类器在一个静态的语言环境中运行,尽管语言和新闻是在一个不断变化的状态。在本文中,我们使用一个新的德国报纸评论数据集来证明,使用朴素ML技术(如随机测试训练分割)训练的分类器在未来的数据上表现不佳,并且时间分层评估分割更合适。我们还表明,分类器的性能迅速下降时,从一个不同的时期比训练数据的数据评估。我们的研究结果表明,在开发滥用语言检测系统时,有必要考虑语言的时间动态,否则部署一个很快就会失效的模型的风险。

英文摘要:

The spread of online hate has become a significant problem for newspapers that host comment sections. As a result, there is growing interest in using machine learning and natural language processing for (semi-) automated abusive language detection to avoid manual comment moderation costs or having to shut down comment sections altogether. However, much of the past work on abusive language detection assumes that classifiers operate in a static language environment, despite language and news being in a state of constant flux. In this paper, we show using a new German newspaper comments dataset that the classifiers trained with naive ML techniques like a random-test train split will underperform on future data, and that a time stratified evaluation split is more appropriate. We also show that classifier performance rapidly degrades when evaluated on data from a different period than the training data. Our findings suggest that it is necessary to consider the temporal dynamics of language when developing an abusive language detection system or risk deploying a model that will quickly become defunct.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑