AI-UPV at IberLEF-2021 DETOXIS task：Toxicity Detection in Immigration-Related Web News Comments Using Transformers and Statistical Models
AI-UPV at IberLEF-2021 DETOXIS task: Toxicity Detection in Immigration-Related Web News Comments Using Transformers and Statistical Models
摘要：
本文介绍了我们在第三届伊比利亚语言评估论坛研讨会上参与西班牙语评论中的毒性检测（DETOXIS）共享任务2021。共享任务分为两个相关的分类任务：（i）任务1：毒性检测;（ii）任务2：毒性水平检测。他们关注的是由于与移民有关的各种在线新闻文章中发表的有毒评论的传播而加剧的仇外问题。缓解这一问题的必要努力之一是检测评论中的毒性。我们的主要目标是实现一个准确的模型，以检测DETOXIS共享任务2021中关于网络新闻文章的评论中的仇外心理，基于竞争的官方指标：任务1的F1分数和任务2的亲密度评估指标（CEM）。为了解决这些任务，我们使用了两种类型的机器学习模型：（i）统计模型和（ii）用于语言理解的深度双向Transformers（BERT）模型。我们在这两个任务中使用BETO获得了最好的结果，BETO是一个在大型西班牙语语料库上训练的BERT模型。我们以0.5996的F1分数获得了任务1官方排名的第三名，并以0.7142的CEM获得了任务2官方排名的第六名。我们的研究结果表明：（i）BERT模型在文本评论中的毒性检测方面比统计模型获得更好的结果;（ii）单语BERT模型在其预训练语言的文本评论中的毒性检测方面比多语言BERT模型具有优势。
英文摘要：
This paper describes our participation in the DEtection of TOXicity in comments In Spanish (DETOXIS) shared task 2021 at the 3rd Workshop on Iberian Languages Evaluation Forum. The shared task is divided into two related classification tasks: (i) Task 1: toxicity detection and; (ii) Task 2: toxicity level detection. They focus on the xenophobic problem exacerbated by the spread of toxic comments posted in different online news articles related to immigration. One of the necessary efforts towards mitigating this problem is to detect toxicity in the comments. Our main objective was to implement an accurate model to detect xenophobia in comments about web news articles within the DETOXIS shared task 2021, based on the competition's official metrics: the F1-score for Task 1 and the Closeness Evaluation Metric (CEM) for Task 2. To solve the tasks, we worked with two types of machine learning models: (i) statistical models and (ii) Deep Bidirectional Transformers for Language Understanding (BERT) models. We obtained our best results in both tasks using BETO, an BERT model trained on a big Spanish corpus. We obtained the 3rd place in Task 1 official ranking with the F1-score of 0.5996, and we achieved the 6th place in Task 2 official ranking with the CEM of 0.7142. Our results suggest: (i) BERT models obtain better results than statistical models for toxicity detection in text comments; (ii) Monolingual BERT models have an advantage over multilingual BERT models in toxicity detection in text comments in their pre-trained language.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
