攻击性语言检测的OSACT4共享任务:基于强化预处理的方法
OSACT4 Shared Task on Offensive Language Detection: Intensive Preprocessing-Based Approach
摘要:
预处理阶段是文本分类管道中的关键阶段之一。本研究的目的是调查文本分类的预处理阶段的影响,特别是对攻击性语言和仇恨言论的阿拉伯语文本分类。社交媒体中使用的阿拉伯语是非正式的,使用阿拉伯方言书写,这使得文本分类任务非常复杂。预处理有助于降维和删除无用的内容。在进一步处理数据集并将其输入分类模型之前,我们对数据集应用了密集的预处理技术。一个密集的预处理为基础的方法表明其显着的影响,攻击性语言检测和仇恨言论检测共享任务的第四次研讨会上的开源阿拉伯语语料库和语料库处理工具(OSACT)。我们的团队在子任务A攻击性语言检测部分获得第三名(第三名),在子任务B仇恨语音检测部分获得第一名(第一名),F1得分分别为89%和95%,通过在阿拉伯语仇恨语音检测的F1,准确性,召回率和精确度方面提供最先进的性能。
英文摘要:
The preprocessing phase is one of the key phases within the text classification pipeline. This study aims at investigating the impact of the preprocessing phase on text classification, specifically on offensive language and hate speech classification for Arabic text. The Arabic language used in social media is informal and written using Arabic dialects, which makes the text classification task very complex. Preprocessing helps in dimensionality reduction and removing useless content. We apply intensive preprocessing techniques to the dataset before processing it further and feeding it into the classification model. An intensive preprocessing-based approach demonstrates its significant impact on offensive language detection and hate speech detection shared tasks of the fourth workshop on Open-Source Arabic Corpora and Corpora Processing Tools (OSACT). Our team wins the third place (3rd) in the Sub-Task A Offensive Language Detection division and wins the first place (1st) in the Sub-Task B Hate Speech Detection division, with an F1 score of 89% and 95%, respectively, by providing the state-of-the-art performance in terms of F1, accuracy, recall, and precision for Arabic hate speech detection.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。