LangGPS：面向联合多语言指令微调的语言可分性引导数据预选择方法
LangGPS: Language Separability Guided Data Pre-Selection for Joint Multilingual Instruction Tuning
AI总结：
针对多语言指令微调中现有数据选择方法忽略语言内在结构的问题，本文提出轻量两阶段预选择框架LangGPS，基于语言可分性筛选数据，可有效提升多语言训练效果与泛化性。
摘要：
联合多语言指令微调是提升大语言模型（LLMs）多语言指令遵循能力与下游性能的广泛采用方案，但最终获得的多语言能力对训练数据的构成与选择高度敏感。现有选择方法多基于文本质量、多样性或任务相关性等特征，通常忽略多语言数据的内在语言结构。本文提出LangGPS，这是一种由语言可分性引导的轻量两阶段预选择框架，语言可分性用于量化模型表示空间中不同语言样本的可区分程度。LangGPS首先基于可分性分数过滤训练数据，再利用现有选择方法优化数据子集。在6个基准、22种语言上开展的大量实验表明，在现有选择方法基础上应用LangGPS，可提升其在多语言训练中的有效性与泛化性，在理解类任务和低资源语言上提升效果尤为显著。进一步分析显示，高可分性样本有助于形成更清晰的语言边界、支持更快适配，低可分性样本则倾向于充当跨语言对齐的桥梁。此外研究还发现，语言可分性可作为多语言课程学习的有效信号，混合不同可分性水平的样本可获得稳定且具泛化性的性能增益。我们希望本工作为多语言场景下的数据效用提供新视角，支撑更具语言感知能力的大语言模型开发。
英文摘要：
Joint multilingual instruction tuning is a widely adopted approach to improve the multilingual instruction-following ability and downstream performance of large language models (LLMs), but the resulting multilingual capability remains highly sensitive to the composition and selection of the training data. Existing selection methods, often based on features like text quality, diversity, or task relevance, typically overlook the intrinsic linguistic structure of multilingual data. In this paper, we propose LangGPS, a lightweight two-stage pre-selection framework guided by language separability which quantifies how well samples in different languages can be distinguished in the model's representation space. LangGPS first filters training data based on separability scores and then refines the subset using existing selection methods. Extensive experiments across six benchmarks and 22 languages demonstrate that applying LangGPS on top of existing selection methods improves their effectiveness and generalizability in multilingual training, especially for understanding tasks and low-resource languages. Further analysis reveals that highly separable samples facilitate the formation of clearer language boundaries and support faster adaptation, while low-separability samples tend to function as bridges for cross-lingual alignment. Besides, we also find that language separability can serve as an effective signal for multilingual curriculum learning, where interleaving samples with diverse separability levels yields stable and generalizable gains. Together, we hope our work offers a new perspective on data utility in multilingual contexts and support the development of more linguistically informed LLMs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
