合并与征服：通过添加目标语言权重来指导多语言模型
Merge and Conquer: Instructing Multilingual Models by Adding Target Language Weights
AI总结：
本文探讨通过合并语言特定基础模型与指令微调模型，以实现低资源语言的有效指令跟随和多语言能力，减少计算成本并提升性能。
摘要：
大型语言模型（LLMs）仍主要集中在英语上，对低资源语言表现有限。现有适应方法如持续预训练需要大量计算资源。在指令模型中，高质量指令数据也是必需的，这在低资源语言社区中往往难以获得。在这些限制下，模型合并提供了一种轻量级替代方案，但其在低资源环境中的潜力尚未系统探索。本文探讨是否可以通过将指令微调的LLM与语言特定的基础模型合并，从而转移语言知识，避免在更强的指令变体可用时需要语言特定的指令和重复微调。通过覆盖四种伊比利亚语言（巴斯克语、加泰罗尼亚语、加利西亚语和西班牙语）和两个模型家族的实验，我们显示合并能有效实现新语言的指令跟随行为，甚至通过组合多个语言特定模型支持多语言能力。我们的结果表明，模型合并是传统适应方法在低资源语言中的可行且高效替代方案，实现了竞争性性能的同时显著降低了计算成本。
英文摘要：
Large Language Models (LLMs) remain heavily centered on English, with limited performance in low-resource languages. Existing adaptation approaches, such as continual pre-training, demand significant computational resources. In the case of instructed models, high-quality instruction data is also required, both of which are often inaccessible for low-resource language communities. Under these constraints, model merging offers a lightweight alternative, but its potential in low-resource contexts has not been systematically explored. In this work, we explore whether it is possible to transfer language knowledge to an instruction-tuned LLM by merging it with a language-specific base model, thereby eliminating the need of language-specific instructions and repeated fine-tuning processes whenever stronger instructed variants become available. Through experiments covering four Iberian languages (Basque, Catalan, Galician, and Spanish) and two model families, we show that merging enables effective instruction following behavior in new languages and even supports multilingual capability through the combination of multiple language-specific models. Our results indicate that model merging is a viable and efficient alternative to traditional adaptation methods for low-resource languages, achieving competitive performance while greatly reducing computational cost.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
