SOLD：僧伽罗语攻击性语言数据集
SOLD: Sinhala Offensive Language Dataset
AI总结：
本文针对低资源语言僧伽罗语构建了首个大型公开攻击性语言数据集SOLD，含1万条句子级和词元级标注推文，并提出半监督扩展数据集SemiSOLD，以推动该语言的攻击性内容自动检测研究。
摘要：
仇恨言论和网络欺凌等攻击性内容在网络上的广泛传播是一个全球性现象。这引发了人工智能（AI）和自然语言处理（NLP）领域的关注，推动了各种自动检测潜在有害内容系统的开发。这些系统需要标注数据集来训练机器学习（ML）模型。然而，除少数显著例外，该主题的大多数数据集都集中于英语和少数其他高资源语言。因此，攻击性语言识别的研究一直局限于这些语言。本文通过解决僧伽罗语（一种低资源的印度-雅利安语，斯里兰卡超过1700万人使用）中的攻击性语言识别问题来填补这一空白。我们引入了僧伽罗语攻击性语言数据集（SOLD），并在该数据集上进行了多项实验。SOLD是一个人工标注的数据集，包含来自Twitter的10,000条帖子，在句子级和词元级均标注为攻击性和非攻击性，从而提高了ML模型的可解释性。SOLD是首个为僧伽罗语编制的大型公开攻击性语言数据集。我们还引入了SemiSOLD，这是一个更大的数据集，包含超过145,000条僧伽罗语推文，采用半监督方法进行标注。
英文摘要：
The widespread of offensive content online, such as hate speech and cyber-bullying, is a global phenomenon. This has sparked interest in the artificial intelligence (AI) and natural language processing (NLP) communities, motivating the development of various systems trained to detect potentially harmful content automatically. These systems require annotated datasets to train the machine learning (ML) models. However, with a few notable exceptions, most datasets on this topic have dealt with English and a few other high-resource languages. As a result, the research in offensive language identification has been limited to these languages. This paper addresses this gap by tackling offensive language identification in Sinhala, a low-resource Indo-Aryan language spoken by over 17 million people in Sri Lanka. We introduce the Sinhala Offensive Language Dataset (SOLD) and present multiple experiments on this dataset. SOLD is a manually annotated dataset containing 10,000 posts from Twitter annotated as offensive and not offensive at both sentence-level and token-level, improving the explainability of the ML models. SOLD is the first large publicly available offensive language dataset compiled for Sinhala. We also introduce SemiSOLD, a larger dataset containing more than 145,000 Sinhala tweets, annotated following a semi-supervised approach.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
