准备 Vuk'uzenzele 和 ZA-gov-multilingual 南非多语言语料库
Preparing the Vuk'uzenzele and ZA-gov-multilingual South African multilingual corpora
AI总结：
本文构建了包含11种南非官方语言的Vuk'uzenzele和ZA-gov-multilingual多语言政府语料库，利用LASER嵌入生成平行句对，并通过微调大规模多语言预训练模型为9种本土语言提供了NMT基准。
摘要：
本文介绍了两个包含多种南非语言的多语言政府主题语料库。这些语料库是通过收集南非政府报纸（Vuk'uzenzele）以及南非政府演讲（ZA-gov-multilingual）构建的，它们被翻译成了所有11种南非官方语言。这些语料库可用于无数的下游NLP任务。创建这些语料库是为了让研究人员能够研究南非政府出版物中使用的语言，重点是了解南非政府官员如何与其选民进行沟通。在本文中，我们重点介绍了收集、清理和发布这些语料库的过程。我们使用语言无关的句子表示（LASER）嵌入，为神经机器翻译（NMT）任务创建了平行句子语料库。利用这些对齐的句子，我们通过微调一个大规模多语言预训练语言模型，为9种本土语言提供了NMT基准。
英文摘要：
This paper introduces two multilingual government themed corpora in various South African languages. The corpora were collected by gathering the South African Government newspaper (Vuk'uzenzele), as well as South African government speeches (ZA-gov-multilingual), that are translated into all 11 South African official languages. The corpora can be used for a myriad of downstream NLP tasks. The corpora were created to allow researchers to study the language used in South African government publications, with a focus on understanding how South African government officials communicate with their constituents. In this paper we highlight the process of gathering, cleaning and making available the corpora. We create parallel sentence corpora for Neural Machine Translation (NMT) tasks using Language-Agnostic Sentence Representations (LASER) embeddings. With these aligned sentences we then provide NMT benchmarks for 9 indigenous languages by fine-tuning a massively multilingual pre-trained language model.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
