多语言教师：评估语言模型用于多语言合成数据生成
Polyglot Teachers: Evaluating Language Models for Multilingual Synthetic Data Generation
AI总结：
本文评估了多语言教师模型的有效性，通过Polyglot Score衡量数据质量和学生模型表现，发现Gemma 3和Aya Expanse在不同学生模型家族中表现优异，数据质量如提示多样性、长度和响应流畅度对教学效果影响显著。
摘要：
从语言模型（LMs）生成监督微调（SFT）数据以教授小型模型多语言任务已变得越来越常见。然而，教师模型的选择通常是随意的，通常默认选择可用的最大型模型，尽管这些模型在非英语语言中可能有显著的能力差距。这种做法可能导致低质量的合成数据和学生下游性能不佳。在本工作中，我们系统地分析了有效多语言教师的特性。我们通过一种称为Polyglot Score的指标测量数据质量的内在指标，并在6种语言上评估了10个LMs，生成超过140万个SFT示例并训练了240个学生模型。在测试的模型中，Gemma 3 27B和Aya Expanse 32B在不同学生基础模型家族中表现一致有效。进一步分析表明，模型规模本身并不显著预测教师有效性；相反，数据质量如提示多样性、长度和响应流畅度捕捉了内在数据质量的93.3%的方差，并预测学生性能。最后，我们提供了实际建议，包括匹配教师-学生对的模型家族以及从现有提示中翻译或响应，这可以为资源较少的语言带来改进。我们希望我们的工作能促进多语言合成数据和LM开发的数据驱动研究。
英文摘要：
Synthesizing supervised finetuning (SFT) data from language models (LMs) to teach smaller models multilingual tasks has become increasingly common. However, teacher model selection is often ad hoc, typically defaulting to the largest available option, even though such models may have significant capability gaps in non-English languages. This practice can result in poor-quality synthetic data and suboptimal student downstream performance. In this work, we systematically characterize what makes an effective multilingual teacher. We combine intrinsic measures of data quality with extrinsic student model performance in a metric we call Polyglot Score. We evaluate 10 LMs across 6 typologically diverse languages, generating over 1.4M SFT examples and training 240 student models. Our analyses reveal that model scale alone does not significantly predict teacher effectiveness: the most effective teachers we identify are consistently smaller than the largest models evaluated, and their ranking is stable across student base model families. Instead, data qualities such as prompt diversity, length, and response fluency capture 93.3% of the variance in intrinsic data quality and predict student performance. Finally, we provide practical recommendations, including matching the model families of teacher-student pairs and generating responses to existing prompts or translating them from English, which can yield improvements for less-resourced languages. We hope that our work advances data-centric research in multilingual synthetic data and LM development.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
