arXiv 2304.121552023-03-21cs.CLcs.LG

African Stopwords 项目：为非洲语言整理停用词

The African Stopwords project: curating stopwords for African languages

Masakhane

Chris Emezue, Hellina Nigatu, Cynthia Thinwa, Helper Zhou, Shamsuddeen Muhammad, Lerato Louis, Idris Abdulmumin, Samuel Oyerinde, Benjamin Ajibade, Olanrewaju S… 展开作者

Chris Emezue, Hellina Nigatu, Cynthia Thinwa, Helper Zhou, Shamsuddeen Muhammad, Lerato Louis, Idris Abdulmumin, Samuel Oyerinde, Benjamin Ajibade, Olanrewaju Samuel, Oviawe Joshua, Emeka Onwuegbuzia, Handel Emezue, Ifeoluwatayo A. Ige, Atnafu Lambebo Tonja, Chiamaka Chukwuneke, Bonaventure F. P. Dossou, Naome A. Etori, Mbonu Chinedu Emmanuel, Oreen Yousuf, Kaosarat Aina, Davis David

备注：: Accepted at the AfricaNLP workshop at ICLR2022

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

针对非洲低资源语言缺乏标准化停用词的问题，本文介绍 African Stopwords 项目，展示十种非洲语言停用词的整理进展及后续计划。

摘要：

停用词是自然语言处理（NLP）信息检索技术中的基础要素。文本数据预处理中的常见任务之一就是去除停用词。当前，尽管英语等高资源语言受益于多种可用停用词，但非洲大陆诸语言等低资源语言却没有标准化、且可在 NLP 软件包中使用的停用词。非洲语言语境中的停用词研究不足，却能够揭示语言之间交叉影响的信息。African Stopwords 项目旨在研究并整理非洲语言的停用词。本文介绍了我们在十种非洲语言上取得的当前进展，以及该项目的未来计划。

英文摘要：

Stopwords are fundamental in Natural Language Processing (NLP) techniques for information retrieval. One of the common tasks in preprocessing of text data is the removal of stopwords. Currently, while high-resource languages like English benefit from the availability of several stopwords, low-resource languages, such as those found in the African continent, have none that are standardized and available for use in NLP packages. Stopwords in the context of African languages are understudied and can reveal information about the crossover between languages. The \textit{African Stopwords} project aims to study and curate stopwords for African languages. In this paper, we present our current progress on ten African languages as well as future plans for the project.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。