在提升母语性能的同时借助文化与本地知识增强大语言模型的多语言能力
Improving Multilingual Capabilities with Cultural and Local Knowledge in Large Language Models While Enhancing Native Performance
AI总结：
针对大语言模型多语言发展不均的问题，研究团队用含文化本地信息的精选印英指令数据微调多款LLM，推出双语模型Mantra-14B，在不损母语性能、不增模型规模与计算开销的前提下提升了双语表现，并开源相关资源助力低资源语言研究。
摘要：
大语言模型（LLMs）已展现出卓越能力，但其发展主要聚焦于英语及其他高资源语言，导致许多语言的服务供给不足。我们推出了最新的印地语-英语双语大语言模型**Mantra-14B**，其在两种语言上的基准测试分数平均提升约3%，表现优于规模为其两倍的模型。我们利用一个包含48.5万条样本的英语与印地语指令数据组成的精选数据集，对Qwen-2.5-14B-Instruct、Phi-4等模型进行指令微调，以提升它们在英语和印地语上的性能。我们的实验涵盖7种不同参数规模的大语言模型，以及超过140次采用不同英印训练数据比例的训练尝试，结果表明，在不损害母语性能的前提下显著提升多语言性能是可行的。此外，我们的方法避免了词表扩展或架构修改这类资源密集型技术，从而保持了较小的模型规模。我们的结果显示，使用具备文化与本地信息的数据进行适度微调，能够在不产生显著计算开销的情况下弥合性能差距。我们以MIT和Apache许可证发布了训练代码、数据集和模型，以助力针对代表性不足及低资源语言的进一步研究。
英文摘要：
Large Language Models (LLMs) have shown remarkable capabilities, but their development has primarily focused on English and other high-resource languages, leaving many languages underserved. We present our latest Hindi-English bi-lingual LLM \textbf{Mantra-14B} with ~3\% average improvement in benchmark scores over both languages, outperforming models twice its size. Using a curated dataset composed of English and Hindi instruction data of 485K samples, we instruction tuned models such as Qwen-2.5-14B-Instruct and Phi-4 to improve performance over both English and Hindi. Our experiments encompassing seven different LLMs of varying parameter sizes and over 140 training attempts with varying English-Hindi training data ratios demonstrated that it is possible to significantly improve multilingual performance without compromising native performance. Further, our approach avoids resource-intensive techniques like vocabulary expansion or architectural modifications, thus keeping the model size small. Our results indicate that modest fine-tuning with culturally and locally informed data can bridge performance gaps without incurring significant computational overhead. We release our training code, datasets, and models under mit and apache licenses to aid further research towards under-represented and low-resource languages.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
