arXiv 2409.109652024-09-17cs.CLcs.LG

多语言模型在低资源非洲语言上的跨语言迁移

Cross-lingual transfer of multilingual models on low resource African Languages

School of Computer Science and Engineering, Vellore Institute of Technology（韦洛尔理工学院计算机科学与工程学院）；Department of Computer Science, University of Pretoria（比勒陀利亚大学计算机科学系）

Harish Thangaraj, Ananya Chenat, Jaskaran Singh Walia, Vukosi Marivate

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本研究以卢旺达语和隆迪语为对象，基准测试单语言与多语言模型的跨语言迁移能力，对比多种架构性能，发现微调后AfriBERT准确率达88.3%，验证了多语言模型在低资源场景下的强迁移能力。

摘要：

大型多语言模型显著推动了自然语言处理（NLP）研究的发展。然而，其高资源需求以及来自多样化数据源的潜在偏差，引发了人们对其在低资源语言上有效性的担忧。相比之下，在单种语言上训练的单语言模型可能更好地捕捉目标语言的细微差别，有望提供更准确的结果。本研究针对单语言模型和多语言模型，基准测试了从高资源语言到低资源语言的跨语言迁移能力，聚焦于两种班图语：Kinyarwanda（卢旺达语）和Kirundi（隆迪语）。我们评估了基于Transformer的架构，包括Multilingual BERT（mBERT）、AfriBERT和BantuBERTa，以及基于神经网络的架构，如BiGRU、CNN和char-CNN的性能。模型在Kinyarwanda上训练并在Kirundi上测试，同时应用微调来评估性能提升程度和灾难性遗忘情况。微调后，AfriBERT取得了88.3%的最高跨语言准确率，而BiGRU以83.3%的准确率成为表现最佳的神经模型。我们还分析了微调后模型对原语言的遗忘程度。尽管单语言模型仍具竞争力，但本研究表明多语言模型在资源有限的场景下具备强大的跨语言迁移能力。

英文摘要：

Large multilingual models have significantly advanced natural language processing (NLP) research. However, their high resource demands and potential biases from diverse data sources have raised concerns about their effectiveness across low-resource languages. In contrast, monolingual models, trained on a single language, may better capture the nuances of the target language, potentially providing more accurate results. This study benchmarks the cross-lingual transfer capabilities from a high-resource language to a low-resource language for both, monolingual and multilingual models, focusing on Kinyarwanda and Kirundi, two Bantu languages. We evaluate the performance of transformer based architectures like Multilingual BERT (mBERT), AfriBERT, and BantuBERTa against neural-based architectures such as BiGRU, CNN, and char-CNN. The models were trained on Kinyarwanda and tested on Kirundi, with fine-tuning applied to assess the extent of performance improvement and catastrophic forgetting. AfriBERT achieved the highest cross-lingual accuracy of 88.3% after fine-tuning, while BiGRU emerged as the best-performing neural model with 83.3% accuracy. We also analyze the degree of forgetting in the original language post-fine-tuning. While monolingual models remain competitive, this study highlights that multilingual models offer strong cross-lingual transfer capabilities in resource limited settings.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。