PuoBERTa:针对Setswana语精选语言模型的训练与评估
PuoBERTa: Training and evaluation of a curated language model for Setswana
AI总结:
本文提出专为Setswana语定制的掩码语言模型PuoBERTa,构建高质量语料库进行训练,并在词性标注、命名实体识别和新闻分类等任务上验证其有效性,同时发布新数据集与基准。
摘要:
自然语言处理(NLP)在英语等资源丰富的语言上取得了显著进展,但在Setswana等低资源语言上却进展缓慢。本文通过提出PuoBERTa来填补这一空白,这是一种专门为Setswana训练的定制化掩码语言模型。我们介绍了如何收集、整理和准备多样化的单语文本,以生成用于PuoBERTa训练的高质量语料库。在以往Setswana单语资源创建工作的基础上,我们在多个NLP任务上评估了PuoBERTa,包括词性标注(POS)、命名实体识别(NER)和新闻分类。此外,我们引入了一个新的Setswana新闻分类数据集,并使用PuoBERTa提供了初步的基准测试。我们的工作证明了PuoBERTa在促进Setswana等研究不足语言NLP能力方面的有效性,并为未来研究方向铺平了道路。
英文摘要:
Natural language processing (NLP) has made significant progress for well-resourced languages such as English but lagged behind for low-resource languages like Setswana. This paper addresses this gap by presenting PuoBERTa, a customised masked language model trained specifically for Setswana. We cover how we collected, curated, and prepared diverse monolingual texts to generate a high-quality corpus for PuoBERTa's training. Building upon previous efforts in creating monolingual resources for Setswana, we evaluated PuoBERTa across several NLP tasks, including part-of-speech (POS) tagging, named entity recognition (NER), and news categorisation. Additionally, we introduced a new Setswana news categorisation dataset and provided the initial benchmarks using PuoBERTa. Our work demonstrates the efficacy of PuoBERTa in fostering NLP capabilities for understudied languages like Setswana and paves the way for future research directions.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。