无需平行语料的零样本跨语言迁移
Zero-shot Cross-lingual Transfer without Parallel Corpus
AI总结：
针对低资源语言任务数据匮乏、现有跨语言迁移依赖难获取的平行语料或翻译模型的问题，提出含双语任务适配与自训练模块的零样本跨语言迁移方法，无需平行语料/翻译模型即在多任务上取得新SOTA。
摘要：
近年来，尽管预训练语言模型在多语言NLP（自然语言处理）任务上取得了巨大成功，但许多低资源语言任务的训练数据匮乏仍限制了其性能。解决该问题的一种有效方法是将知识从高资源语言迁移到低资源语言。然而，以往许多跨语言迁移研究严重依赖难以获取的平行语料或翻译模型。我们提出一种利用预训练模型实现零样本跨语言迁移的新方法，包含用于执行任务相关双语信息对齐的Bilingual Task Fitting（双语任务适配）模块，以及为无标注数据生成伪软标签和硬标签并利用其开展自训练的自训练模块。我们在不依赖任何平行语料或翻译模型的情况下，在不同任务上取得了新的SOTA（当前最优结果）。
英文摘要：
Recently, although pre-trained language models have achieved great success on multilingual NLP (Natural Language Processing) tasks, the lack of training data on many tasks in low-resource languages still limits their performance. One effective way of solving that problem is to transfer knowledge from rich-resource languages to low-resource languages. However, many previous works on cross-lingual transfer rely heavily on the parallel corpus or translation models, which are often difficult to obtain. We propose a novel approach to conduct zero-shot cross-lingual transfer with a pre-trained model. It consists of a Bilingual Task Fitting module that applies task-related bilingual information alignment; a self-training module generates pseudo soft and hard labels for unlabeled data and utilizes them to conduct self-training. We got the new SOTA on different tasks without any dependencies on the parallel corpus or translation models.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
