CrossIn：一种用于跨语言知识对齐的高效指令微调方法
CrossIn: An Efficient Instruction Tuning Approach for Cross-Lingual Knowledge Alignment
AI总结：
针对LLM多语言训练数据失衡问题，提出CrossIn混合跨语言指令微调方法，并通过多任务多维度基准验证其可显著提升跨语言任务性能与一致性。
摘要：
多语言能力对大型语言模型（LLM）而言是一项重大挑战。以英语为中心的模型在其他语言上通常表现欠佳，尤其是那些与英语语言距离较远的语言。这种性能差异主要源于预训练和指令微调阶段各语言训练数据分布不均衡。为解决这一问题，我们提出了一种名为 CrossIn 的新方法，该方法利用跨语言指令微调数据的混合构成。我们的方法借助不同语言共享的压缩表示，在单一过程中高效提升模型的任务解决能力和多语言能力。此外，我们引入了一个多任务、多维度的基准来评估 CrossIn 的有效性。实验结果表明，我们的方法显著提升了跨任务和跨语言的性能；同时，我们就跨语言数据量以及翻译数据的整合对增强多语言一致性和准确性的影响提供了广泛洞见。
英文摘要：
Multilingual proficiency presents a significant challenge for large language models (LLMs). English-centric models are usually suboptimal in other languages, particularly those that are linguistically distant from English. This performance discrepancy mainly stems from the imbalanced distribution of training data across languages during pre-training and instruction tuning stages. To address this problem, we propose a novel approach called CrossIn, which utilizes a mixed composition of cross-lingual instruction tuning data. Our method leverages the compressed representation shared by various languages to efficiently enhance the model's task-solving capabilities and multilingual proficiency within a single process. In addition, we introduce a multi-task and multi-faceted benchmark to evaluate the effectiveness of CrossIn. Experimental results demonstrate that our method substantially improves performance across tasks and languages, and we provide extensive insights into the impact of cross-lingual data volume and the integration of translation data on enhancing multilingual consistency and accuracy.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
