弥合差距：通过新基准、微调与文化调整提升低资源非洲语言的LLM性能
Bridging the Gap: Enhancing LLM Performance for Low-Resource African Languages with New Benchmarks, Fine-Tuning, and Cultural Adjustments
AI总结：
本研究创建8种低资源非洲语言基准，通过微调、跨语言迁移和文化调整，分别提升LLM性能5.6%、2.9%和3.0%，弥合与英语的差距。
摘要：
大型语言模型（LLMs）在各种任务中表现出色，但在非英语语言，尤其是非洲本土语言方面仍存在显著差距。本文通过创建约100万个人工翻译词语的8种低资源非洲语言新基准数据来解决这些差距，覆盖超过1.6亿使用者，包括：阿姆哈拉语、班巴拉语、伊博语、塞佩迪语（北索托语）、绍纳语、塞索托语（南索托语）、茨瓦纳语和聪加语。我们的基准是Winogrande和MMLU三个部分（大学医学、临床知识和病毒学）的翻译。利用翻译后的基准，我们报告了最先进（SOTA）LLMs在英语和非洲语言之间先前未知的性能差距。最后，利用超过400个微调模型的结果，我们探索了多种缩小LLM性能差距的方法，包括高质量数据集微调（使用LLM作为标注者）、跨语言迁移和文化适宜性调整。关键发现包括：微调平均单语改进5.6%（使用高质量数据相较于低质量数据平均单语改进5.4%），跨语言迁移平均提升2.9%，以及文化适宜性问题上的开箱即用性能提升3.0%。本研究的公开基准、翻译和代码支持进一步研究和开发，旨在创建更具包容性和有效性的语言技术。
英文摘要：
Large Language Models (LLMs) have shown remarkable performance across various tasks, yet significant disparities remain for non-English languages, and especially native African languages. This paper addresses these disparities by creating approximately 1 million human-translated words of new benchmark data in 8 low-resource African languages, covering a population of over 160 million speakers of: Amharic, Bambara, Igbo, Sepedi (Northern Sotho), Shona, Sesotho (Southern Sotho), Setswana, and Tsonga. Our benchmarks are translations of Winogrande and three sections of MMLU: college medicine, clinical knowledge, and virology. Using the translated benchmarks, we report previously unknown performance gaps between state-of-the-art (SOTA) LLMs in English and African languages. Finally, using results from over 400 fine-tuned models, we explore several methods to reduce the LLM performance gap, including high-quality dataset fine-tuning (using an LLM-as-an-Annotator), cross-lingual transfer, and cultural appropriateness adjustments. Key findings include average mono-lingual improvements of 5.6% with fine-tuning (with 5.4% average mono-lingual improvements when using high-quality data over low-quality data), 2.9% average gains from cross-lingual transfer, and a 3.0% out-of-the-box performance boost on culturally appropriate questions. The publicly available benchmarks, translations, and code from this study support further research and development aimed at creating more inclusive and effective language technologies.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
