机器翻译能否桥接多语言预训练与跨语言迁移学习？
Can Machine Translation Bridge Multilingual Pretraining and Cross-lingual Transfer Learning?
AI总结：
本文研究了将机器翻译作为持续训练目标以桥接多语言预训练与跨语言应用的潜力，发现显式句子级对齐不仅未能增强跨语言表征学习，反而因输出可分性对跨语言迁移预训练产生有害影响。
摘要：
多语言预训练与微调在各种自然语言处理任务中取得了显著成功。将表征从一种语言迁移到另一种语言对于跨语言学习尤为关键。人们可以期望机器翻译目标非常适合培养这种能力，因为它们涉及对不同语言中语义等价句子的显式对齐。本文研究了采用机器翻译作为持续训练目标以增强语言表征学习的潜在益处，从而桥接多语言预训练与跨语言应用。我们通过两个视角研究这一问题：对现有模型性能的定量评估，以及对其潜在表征的分析。我们的结果表明，与预期相反，作为持续训练的机器翻译在多个跨语言自然语言理解任务中未能增强跨语言表征学习。我们得出结论，跨语言场景中的显式句子级对齐对跨语言迁移预训练是有害的，这对未来的跨语言迁移研究具有重要意义。我们进一步通过相似性度量和参数调查提供证据，表明这种缺乏积极影响的原因在于输出可分性——我们认为这对机器翻译有用，但在其他地方是有害的。
英文摘要：
Multilingual pretraining and fine-tuning have remarkably succeeded in various natural language processing tasks. Transferring representations from one language to another is especially crucial for cross-lingual learning. One can expect machine translation objectives to be well suited to fostering such capabilities, as they involve the explicit alignment of semantically equivalent sentences from different languages. This paper investigates the potential benefits of employing machine translation as a continued training objective to enhance language representation learning, bridging multilingual pretraining and cross-lingual applications. We study this question through two lenses: a quantitative evaluation of the performance of existing models and an analysis of their latent representations. Our results show that, contrary to expectations, machine translation as the continued training fails to enhance cross-lingual representation learning in multiple cross-lingual natural language understanding tasks. We conclude that explicit sentence-level alignment in the cross-lingual scenario is detrimental to cross-lingual transfer pretraining, which has important implications for future cross-lingual transfer studies. We furthermore provide evidence through similarity measures and investigation of parameters that this lack of positive influence is due to output separability -- which we argue is of use for machine translation but detrimental elsewhere.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
