指令微调中跨语言零样本泛化的深度探索
Deep Exploration of Cross-Lingual Zero-Shot Generalization in Instruction Tuning
AI总结：
本文通过英语与韩语元数据集的指令微调实验，并引入含51个基准的KORANI及跨语言模板，证明跨语言零样本泛化可显著提升未见任务表现。
摘要：
指令微调已成为一种强大的技术，显著提升了在未见任务上的零样本性能。尽管近期工作通过将指令微调应用于多语言模型来探索跨语言泛化，但先前研究主要聚焦英语，对非英语任务的探索有限。为深入研究指令微调中的跨语言泛化，我们分别在两个不同语言的元数据集上进行指令微调。随后，我们在与训练所用语言不同的语言上评估未见任务的性能。为支持该研究，我们引入了一个名为“KORANI”（Korean Natural Instruction，韩国自然指令）的新型非英语元数据集，包含51个韩语基准。此外，我们设计了跨语言模板，以缓解跨语言设置下训练与推理之间模板在语言和指令格式上的差异。实验表明，英语和韩语均通过跨语言泛化获得持续提升，平均得分分别比基线高出20.7%和13.6%。值得注意的是，这些提升与单语指令微调取得的效果相当，甚至在某些任务上超过后者。结果强调，在指令微调期间，跨语言获取相关数据比与未见任务保持语言一致性更重要。
英文摘要：
Instruction tuning has emerged as a powerful technique, significantly boosting zero-shot performance on unseen tasks. While recent work has explored cross-lingual generalization by applying instruction tuning to multilingual models, previous studies have primarily focused on English, with a limited exploration of non-English tasks. For an in-depth exploration of cross-lingual generalization in instruction tuning, we perform instruction tuning individually for two distinct language meta-datasets. Subsequently, we assess the performance on unseen tasks in a language different from the one used for training. To facilitate this investigation, we introduce a novel non-English meta-dataset named "KORANI" (Korean Natural Instruction), comprising 51 Korean benchmarks. Moreover, we design cross-lingual templates to mitigate discrepancies in language and instruction-format of the template between training and inference within the cross-lingual setting. Our experiments reveal consistent improvements through cross-lingual generalization in both English and Korean, outperforming baseline by average scores of 20.7\% and 13.6\%, respectively. Remarkably, these enhancements are comparable to those achieved by monolingual instruction tuning and even surpass them in some tasks. The result underscores the significance of relevant data acquisition across languages over linguistic congruence with unseen tasks during instruction tuning.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
