arXiv 2310.022492023-10-03cs.CLcs.LG

利用预训练句子Transformer进行印度语言的攻击性语言检测

Harnessing Pre-Trained Sentence Transformers for Offensive Language Detection in Indian Languages

MKSSS Cummins College of Engineering for Women（MKSSS卡明斯女子工程学院）；Indian Institute of Technology Madras（印度理工学院马德拉斯分校）；Peoples' Friendship University of Russia (RUDN University)（俄罗斯人民友谊大学（RUDN大学））；Joint Institute for Nuclear Research（联合核研究所）；Vrije Universiteit Amsterdam（阿姆斯特丹自由大学）；University of Skövde（舍夫德大学）

Ananya Joshi, Raviraj Joshi

备注：: HASOC at FIRE 2023

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本研究针对孟加拉语、阿萨姆语、古吉拉特语三种低资源印度语言的仇恨言论检测任务，基于HASOC 2023数据集微调BERT与SBERT模型，发现单语sentence-BERT模型表现最优，为构建包容性网络空间提供支持。

摘要：

在我们日益互联的数字世界中，社交媒体平台已成为传播仇恨言论和攻击性内容的强大渠道。本工作深入研究仇恨言论检测领域，重点关注三种低资源印度语言：孟加拉语、阿萨姆语和古吉拉特语。该挑战被构建为文本分类任务，旨在辨别一条推文是否包含攻击性或非攻击性内容。我们利用HASOC 2023数据集，对预训练BERT和SBERT模型进行微调，以评估它们识别仇恨言论的有效性。我们的研究结果凸显了单语sentence-BERT模型的优越性，尤其是在孟加拉语中，我们取得了最高排名。然而，阿萨姆语和古吉拉特语的表现表明仍有提升空间。我们的目标是通过遏制仇恨言论的扩散，打造具有包容性的网络空间。

英文摘要：

In our increasingly interconnected digital world, social media platforms have emerged as powerful channels for the dissemination of hate speech and offensive content. This work delves into the domain of hate speech detection, placing specific emphasis on three low-resource Indian languages: Bengali, Assamese, and Gujarati. The challenge is framed as a text classification task, aimed at discerning whether a tweet contains offensive or non-offensive content. Leveraging the HASOC 2023 datasets, we fine-tuned pre-trained BERT and SBERT models to evaluate their effectiveness in identifying hate speech. Our findings underscore the superiority of monolingual sentence-BERT models, particularly in the Bengali language, where we achieved the highest ranking. However, the performance in Assamese and Gujarati languages signifies ongoing opportunities for enhancement. Our goal is to foster inclusive online spaces by countering hate speech proliferation.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。