一种研究多语言语言模型中跨语言迁移的高效方法
An Efficient Approach for Studying Cross-Lingual Transfer in Multilingual Language Models
AI总结：
本文提出一种基于适配器解耦任务与语言的高效方法，系统研究多语言模型中迁移语言对目标语言零样本性能的影响，量化负向干扰并给出有益配置列表。
摘要：
预训练多语言模型（MLMs）在零样本跨语言迁移方面的能力和有效性已得到充分证实。然而，正向或负向迁移现象以及语言选择的影响仍有待充分理解，尤其是在大规模多语言语言模型的复杂环境中。我们提出了一种高效的方法，用于研究迁移语言对另一目标语言零样本性能的影响。与以往工作不同，我们的方法通过使用专用的适配器单元，将下游任务与语言解耦。我们的研究结果表明，一些语言对其他语言影响不大，而另一些语言，尤其是预训练期间未见过的语言，对不同的目标语言可能极为有益或有害。我们发现没有任何一种迁移语言对所有目标语言都有益。有趣的是，我们观察到MLMs之前未见过的语言始终能从几乎任何语言的迁移中受益。此外，我们利用模块化方法高效地量化负向干扰，并据此对语言进行分类。我们还提供了一份有前景的迁移-目标语言配置列表，这些配置能够持续提升目标语言的性能。代码和数据已公开：https://github.com/ffaisal93/neg_inf
英文摘要：
The capacity and effectiveness of pre-trained multilingual models (MLMs) for zero-shot cross-lingual transfer is well established. However, phenomena of positive or negative transfer, and the effect of language choice still need to be fully understood, especially in the complex setting of massively multilingual LMs. We propose an \textit{efficient} method to study transfer language influence in zero-shot performance on another target language. Unlike previous work, our approach disentangles downstream tasks from language, using dedicated adapter units. Our findings suggest that some languages do not largely affect others, while some languages, especially ones unseen during pre-training, can be extremely beneficial or detrimental for different target languages. We find that no transfer language is beneficial for all target languages. We do, curiously, observe languages previously unseen by MLMs consistently benefit from transfer from almost any language. We additionally use our modular approach to quantify negative interference efficiently and categorize languages accordingly. Furthermore, we provide a list of promising transfer-target language configurations that consistently lead to target language performance improvements. Code and data are publicly available: https://github.com/ffaisal93/neg_inf
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
