SERENGETI：面向非洲的大规模多语言语言模型
SERENGETI: Massively Multilingual Language Models for Africa
AI总结：
本文提出SERENGETI，一个覆盖517种非洲语言的大规模多语言预训练模型，在八项自然语言理解任务上超越现有模型，并分析了零样本设置下语言谱系与相似性对错误的影响。
摘要：
多语言预训练语言模型（mPLMs）在预训练过程中获取了有价值且可泛化的语言信息，并推动了特定任务微调的最新技术水平。迄今为止，现有语言模型仅覆盖了约2000种非洲语言中的约31种。我们通过开发SERENGETI来改善这一局限性，这是一个覆盖517种非洲语言及语言变体的大规模多语言语言模型。我们在20个数据集上的八项自然语言理解任务中评估了我们的新模型，并与覆盖4-23种非洲语言的4个mPLMs进行比较。SERENGETI在八项任务中的11个数据集上优于其他模型，取得了82.27的平均F_1分数。我们还对模型产生的错误进行了分析，这使我们能够在零样本设置下应用模型时，研究语言谱系和语言相似性的影响。我们将公开发布我们的模型以供研究使用。
英文摘要：
Multilingual pretrained language models (mPLMs) acquire valuable, generalizable linguistic information during pretraining and have advanced the state of the art on task-specific finetuning. To date, only ~31 out of ~2,000 African languages are covered in existing language models. We ameliorate this limitation by developing SERENGETI, a massively multilingual language model that covers 517 African languages and language varieties. We evaluate our novel models on eight natural language understanding tasks across 20 datasets, comparing to 4 mPLMs that cover 4-23 African languages. SERENGETI outperforms other models on 11 datasets across the eights tasks, achieving 82.27 average F_1. We also perform analyses of errors from our models, which allows us to investigate the influence of language genealogy and linguistic similarity when the models are applied under zero-shot settings. We will publicly release our models for research.\footnote{\href{https://github.com/UBC-NLP/serengeti}{https://github.com/UBC-NLP/serengeti}}
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
