将英语中心的大语言模型转变为多语言者：需要多少多语言性？
Turning English-centric LLMs Into Polyglots: How Much Multilinguality Is Needed?
AI总结：
本文研究英语中心大语言模型实现跨语言泛化所需的最少多语言微调量，发现仅用两到三种语言进行多语言指令微调即可有效激发跨语言能力，且对生成式任务尤其有益。
摘要：
当今绝大多数大语言模型（LLMs）以英语为中心，主要在英语文本上进行预训练。然而，为了满足用户期望，模型一旦部署到下游应用中，就需要能够以多种语言做出适当回应。这需要强大的跨语言迁移能力。在这项工作中，我们研究了在微调期间引发英语中心大语言模型跨语言泛化所需的最少多语言性。在四个大语言模型上的实验中，我们发现仅使用两到三种语言进行多语言指令微调，既是必要的，也是足以引发有效跨语言泛化的，其限制因素在于目标语言在预训练期间被看到的程度。在五个不同任务上的评估进一步揭示，多语言指令微调对于假设输入/输出语言一致的生成式任务（例如聊天场景）最为有益，而对于高度结构化的分类式任务则重要性较低。我们的代码和数据可在 https://github.com/ZurichNLP/multilingual-instruction-tuning 获取。
英文摘要：
The vast majority of today's large language models (LLMs) are English-centric, having been pretrained predominantly on English text. Yet, in order to meet user expectations, models need to be able to respond appropriately in multiple languages once deployed in downstream applications. This requires strong cross-lingual transfer abilities. In this work, we investigate the minimal amount of multilinguality required during finetuning to elicit cross-lingual generalisation in English-centric LLMs. In experiments across four LLMs, we find that multilingual instruction tuning with as few as two to three languages is both necessary and sufficient to elicit effective cross-lingual generalisation, with the limiting factor being the degree to which a target language is seen during pretraining. Evaluations on five different tasks further reveal that multilingual instruction tuning is most beneficial for generative tasks that assume input/output language agreement, such as in chat settings, while being of less importance for highly structured classification-style tasks. Our code and data is available at https://github.com/ZurichNLP/multilingual-instruction-tuning.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
