arXiv 2501.082842025-01-15cs.CL

AfriHate：面向非洲语言的仇恨言论与辱骂性语言多语言数据集集合

AfriHate: A Multilingual Collection of Hate Speech and Abusive Language Datasets for African Languages

Imperial College London（伦敦帝国学院）；Bayero University Kano（卡诺贝罗大学）；University of Pretoria（比勒陀利亚大学）；Bahir Dar University（巴赫达尔大学）；University of Hamburg（汉堡大学）；Mila（米拉研究所）；McGill University（麦吉尔大学）；Northeastern University（东北大学）；Maseno University（马塞诺大学）；Digital Umuganda（数字乌干达）；Haramaya University（哈拉马亚大学）；Al Akhawayn University（阿赫瓦因大学）；Uppsala University（乌普萨拉大学）；Istanbul Technical University（伊斯坦布尔理工大学）；SADiLaR（南非洲数字语言资源中心）；University of Deusto（德乌斯托大学）；Instituto Politécnico Nacional（国家理工学院）；Wollo University（沃洛大学）；Addis Ababa University（亚的斯亚贝巴大学）；Lancaster University（兰卡斯特大学）；Bocconi University（博科尼大学）；Cardiff University（卡迪夫大学）

Shamsuddeen Hassan Muhammad, Idris Abdulmumin, Abinew Ali Ayele, David Ifeoluwa Adelani, Ibrahim Said Ahmad, Saminu Mohammad Aliyu, Nelson Odhiambo Onyango, Lil… 展开作者

Shamsuddeen Hassan Muhammad, Idris Abdulmumin, Abinew Ali Ayele, David Ifeoluwa Adelani, Ibrahim Said Ahmad, Saminu Mohammad Aliyu, Nelson Odhiambo Onyango, Lilian D. A. Wanzare, Samuel Rutunda, Lukman Jibril Aliyu, Esubalew Alemneh, Oumaima Hourrane, Hagos Tesfahun Gebremichael, Elyas Abdi Ismail, Meriem Beloucif, Ebrahim Chekol Jibril, Andiswa Bukula, Rooweither Mabuya, Salomey Osei, Abigail Oppong, Tadesse Destaw Belay, Tadesse Kebede Guge, Tesfa Tegegne Asfaw, Chiamaka Ijeoma Chukwuneke, Paul Röttger, Seid Muhie Yimam, Nedjma Ousidhoum

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

针对全球南方非洲语言仇恨言论管控因缺乏本地高质量数据和社区参与而失效的问题，研究构建了含15种非洲语言、由母语者标注的AfriHate多语言仇恨言论数据集，给出了LLM相关分类基线并开源资源。

摘要：

仇恨言论和辱骂性语言是全球性现象，需要社会文化背景知识才能理解、识别和管控。然而在全球南方的许多地区，已有多起记录的问题：一是缺乏内容管控，二是因依赖脱离语境的关键词检测而导致的审查问题。此外，公众人物经常成为管控流程的核心对象，而针对少数群体的大规模定向仇恨言论宣传却被忽视。这些局限主要源于当地语言高质量数据的匮乏，以及数据收集、标注和管控流程未纳入本地社区。为解决这一问题，我们推出AfriHate：一个包含15种非洲语言的仇恨言论与辱骂性语言多语言数据集集合。AfriHate中的每条样本都由熟悉本地文化的母语使用者标注。我们报告了数据集构建相关的挑战，并给出了使用和不使用大语言模型（LLMs）的多种分类基线结果。数据集、个体标注结果以及仇恨言论和冒犯性语言词典可在https://github.com/AfriHate/AfriHate获取。

英文摘要：

Hate speech and abusive language are global phenomena that need socio-cultural background knowledge to be understood, identified, and moderated. However, in many regions of the Global South, there have been several documented occurrences of (1) absence of moderation and (2) censorship due to the reliance on keyword spotting out of context. Further, high-profile individuals have frequently been at the center of the moderation process, while large and targeted hate speech campaigns against minorities have been overlooked. These limitations are mainly due to the lack of high-quality data in the local languages and the failure to include local communities in the collection, annotation, and moderation processes. To address this issue, we present AfriHate: a multilingual collection of hate speech and abusive language datasets in 15 African languages. Each instance in AfriHate is annotated by native speakers familiar with the local culture. We report the challenges related to the construction of the datasets and present various classification baseline results with and without using LLMs. The datasets, individual annotations, and hate speech and offensive language lexicons are available on https://github.com/AfriHate/AfriHate

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。