通过带翻译指令的多语言微调激发大语言模型的翻译能力
Eliciting the Translation Ability of Large Language Models via Multilingual Finetuning with Translation Instructions
AI总结：
本研究通过对XGLM-7B进行带翻译指令的多语言微调与分析，揭示多语言LLM翻译能力的影响因素与指令执行机制，证实微调可提升其对未见语言对的翻译表现。
摘要：
大规模预训练语言模型（LLM），如ChatGPT和GPT4，即使未在平行语料上接受显式训练，也已展现出强大的多语言翻译能力。LLM是如何获得执行不同语言翻译指令的能力，这一问题值得探究。本文通过对多语言预训练语言模型XGLM-7B进行微调，使其遵循给定指令执行多语言翻译，展开了详细分析。首先，我们发现多语言LLM的翻译能力比此前已证实的更强；对于特定语言，其翻译性能取决于该语言与英语的相似度以及预训练阶段所用的数据量。其次，我们发现LLM执行翻译指令的能力依赖于对翻译指令的理解以及不同语言之间的对齐。通过多语言微调，LLM甚至能够很好地完成指令微调阶段未见过的语言对的翻译任务。
英文摘要：
Large-scale Pretrained Language Models (LLMs), such as ChatGPT and GPT4, have shown strong abilities in multilingual translations, without being explicitly trained on parallel corpora. It is interesting how the LLMs obtain their ability to carry out translation instructions for different languages. In this paper, we present a detailed analysis by finetuning a multilingual pretrained language model, XGLM-7B, to perform multilingual translation following given instructions. Firstly, we show that multilingual LLMs have stronger translation abilities than previously demonstrated. For a certain language, the performance depends on its similarity to English and the amount of data used in the pretraining phase. Secondly, we find that LLMs' ability to carry out translation instructions relies on the understanding of translation instructions and the alignment among different languages. With multilingual finetuning, LLMs could learn to perform the translation task well even for those language pairs unseen during the instruction tuning phase.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
