自增强提升零样本跨语言迁移
Self-Augmentation Improves Zero-Shot Cross-Lingual Transfer
AI总结：
本文提出自增强方法SALT，通过语码转换和嵌入混合从多语言预训练模型中蒸馏跨语言知识，无需外部对齐数据即可提升零样本跨语言迁移能力。
摘要：
零样本跨语言迁移是多语言自然语言处理中的核心任务，它使在训练资源较充足的语言上训练的模型能够泛化到其他低资源语言。此前针对该任务的研究通常利用平行语料库、双语词典或其他带标注的对齐数据来提升跨语言迁移能力，但这些数据的获取通常代价高昂。本文提出了一种简单而有效的方法 SALT，无需借助此类外部数据即可提升多语言预训练语言模型的零样本跨语言迁移能力。通过将语码转换和嵌入混合与自增强相结合，SALT 能够有效地从多语言预训练语言模型中蒸馏跨语言知识，并增强其在下游任务上的迁移能力。在 XNLI 和 PAWS-X 上的实验结果表明，我们的方法能够在没有外部数据的情况下提升零样本跨语言迁移能力。我们的代码可在 https://github.com/luka-group/SALT 获取。
英文摘要：
Zero-shot cross-lingual transfer is a central task in multilingual NLP, allowing models trained in languages with more sufficient training resources to generalize to other low-resource languages. Earlier efforts on this task use parallel corpora, bilingual dictionaries, or other annotated alignment data to improve cross-lingual transferability, which are typically expensive to obtain. In this paper, we propose a simple yet effective method, SALT, to improve the zero-shot cross-lingual transfer of the multilingual pretrained language models without the help of such external data. By incorporating code-switching and embedding mixup with self-augmentation, SALT effectively distills cross-lingual knowledge from the multilingual PLM and enhances its transferability on downstream tasks. Experimental results on XNLI and PAWS-X show that our method is able to improve zero-shot cross-lingual transferability without external data. Our code is available at https://github.com/luka-group/SALT.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
