多少种语言适合进行多语言指令微调？基于BLOOM的案例研究
How Many Languages Make Good Multilingual Instruction Tuning? A Case Study on BLOOM
AI总结：
该研究以BLOOM为对象开展多语言指令微调案例研究，探究语言数量、接触量、语际相似性对性能的影响，证实扩大语言覆盖有益，测试语言在训练集中可显著提效，且谱系特征比语言数量更影响跨语言迁移。
摘要：
使用多语言对大语言模型进行指令微调，可使其适配多语言下游任务。但目前仍不确定：仅用少量语言是否足够，还是加入更多语言能持续提升效果。我们通过在1到52种语言上微调大型多语言模型，开展了一项基于BLOOM的案例研究，以探究影响性能的三个关键因素：语言数量、语言接触量，以及训练与测试语言间的相似性。总体而言我们发现：1）在多语言指令微调中扩大语言覆盖范围是有益的；2）若测试语言出现在指令混合语料中，准确率通常会显著提升；3）语言的谱系特征与跨语言迁移的相关性，比单纯的语言数量更强，但不同语言受益程度各异。
英文摘要：
Instruction tuning a large language model with multiple languages can prepare it for multilingual downstream tasks. Nonetheless, it is yet to be determined whether having a handful of languages is sufficient, or whether the benefits increase with the inclusion of more. By fine-tuning large multilingual models on 1 to 52 languages, we present a case study on BLOOM to understand three pertinent factors affecting performance: the number of languages, language exposure, and similarity between training and test languages. Overall we found that 1) expanding language coverage in multilingual instruction tuning proves to be beneficial; 2) accuracy often significantly boots if the test language appears in the instruction mixture; 3) languages' genetic features correlate with cross-lingual transfer more than merely the number of language but different languages benefit to various degrees.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
