Investigating an approach for low resource language dataset creation, curation and classification: Setswana and Sepedi

Training Cross-Lingual embeddings for Setswana and Sepedi

African languages still lag in the advances of Natural Language Processing techniques, one reason being the lack of representative data, having a technique that can transfer information between languages can help mitigate against the lack of data problem. This paper trains Setswana and Sepedi monolingual word vectors and uses VecMap to create cross-lingual embeddings for Setswana-Sepedi in order to do a cross-lingual transfer. Word embeddings are word vectors that represent words as continuous floating numbers where semantically similar words are mapped to nearby points in n-dimensional space.…

Low resource language dataset creation, curation and classification: Setswana and Sepedi -- Extended Abstract

The recent advances in Natural Language Processing have only been a boon for well represented languages, negating research in lesser known global languages. This is in part due to the availability of curated data and research resources. One of the current challenges concerning low-resourced languages are clear guidelines on the collection, curation and preparation of datasets for different use-cases. In this work, we take on the task of creating two datasets that are focused on news headlines (i.e short text) for Setswana and Sepedi and the creation of a news topic classification task from the…

语言无关的基于远监督的情感标注：以英语、塞佩迪语和塞斯瓦纳语为例

Language-Independent Sentiment Labelling with Distant Supervision: A Case Study for English, Sepedi and Setswana

本文提出一种语言无关的情感标注方法，利用emoji和词语信息，对英语、塞佩迪语和塞斯瓦纳语的推文进行情感标注，准确率分别达到66%、69%和63%。

弥合差距：通过新基准、微调与文化调整提升低资源非洲语言的LLM性能

Bridging the Gap: Enhancing LLM Performance for Low-Resource African Languages with New Benchmarks, Fine-Tuning, and Cultural Adjustments

本研究创建8种低资源非洲语言基准，通过微调、跨语言迁移和文化调整，分别提升LLM性能5.6%、2.9%和3.0%，弥合与英语的差距。