PLUG：在跨语言指令微调中利用枢纽语言
PLUG: Leveraging Pivot Language in Cross-Lingual Instruction Tuning
AI总结：
提出PLUG方法，利用高资源语言作为枢纽增强低资源语言的指令微调，并引入X-AlpacaEval基准，使LLMs指令遵循能力平均提升29%。
摘要：
指令微调在理解和响应多样化人类指令方面显著推进了大型语言模型（LLMs）。尽管在高资源语言中取得了成功，但由于LLMs在不同语言间的基础能力不平衡（源于其预训练数据中语言分布不均），其在低资源语言中的应用面临挑战。为解决此问题，我们提出枢纽语言引导生成（PLUG），该方法利用高资源语言（主要是英语）作为枢纽，以增强低资源语言的指令微调。它训练模型首先用枢纽语言处理指令，然后用目标语言生成响应。为评估该方法，我们引入了一个基准X-AlpacaEval，包含4种语言（中文、韩语、意大利语和西班牙语）的指令，每种均由专业翻译人员标注。与仅直接用目标语言响应相比，我们的方法使LLMs的指令遵循能力平均显著提升了29%。进一步实验验证了该方法的通用性，即通过使用英语以外的替代枢纽语言来辅助LLMs熟练度较低的语言。
英文摘要：
Instruction tuning has remarkably advanced large language models (LLMs) in understanding and responding to diverse human instructions. Despite the success in high-resource languages, its application in lower-resource ones faces challenges due to the imbalanced foundational abilities of LLMs across different languages, stemming from the uneven language distribution in their pre-training data. To tackle this issue, we propose pivot language guided generation (PLUG), an approach that utilizes a high-resource language, primarily English, as the pivot to enhance instruction tuning in lower-resource languages. It trains the model to first process instructions in the pivot language, and then produce responses in the target language. To evaluate our approach, we introduce a benchmark, X-AlpacaEval, of instructions in 4 languages (Chinese, Korean, Italian, and Spanish), each annotated by professional translators. Our approach demonstrates a significant improvement in the instruction-following abilities of LLMs by 29% on average, compared to directly responding in the target language alone. Further experiments validate the versatility of our approach by employing alternative pivot languages beyond English to assist languages where LLMs exhibit lower proficiency. Our code and data are available at https://github.com/ytyz1307zzh/PLUG.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
