通过多语言微调和反向翻译实现多语言双向无监督翻译
Multilingual Bidirectional Unsupervised Translation Through Multilingual Finetuning and Back-Translation
AI总结：
提出两阶段方法EcXTra，通过多语言微调与反向翻译训练单一NMT模型，实现未见语言与英语间的双向无监督翻译，在7种低资源语言上取得竞争性表现并刷新英译哈萨克语纪录。
摘要：
我们提出了一种两阶段方法，用于训练单个 NMT 模型以在未见语言与英语之间进行双向翻译。在第一阶段，我们将编码器-解码器模型初始化为预训练的 XLM-R 和 RoBERTa 权重，然后在 40 种语言到英语的并行数据上执行多语言微调。我们发现该模型能够泛化到对未见语言的零样本翻译。在第二阶段，我们利用这种泛化能力从单语数据集生成合成并行数据，然后通过连续轮次的反向翻译进行双向训练。我们的方法名为 EcXTra（以英语为中心的跨语言（X）迁移），在概念上很简单，全程仅使用标准的交叉熵目标。它也是数据驱动的，依次利用辅助并行数据和单语数据。我们评估了 7 种低资源语言的无监督 NMT 结果，发现每一轮反向翻译训练都会进一步优化双向性能。我们最终训练的单一 EcXTra 模型在所有翻译方向上均实现了具有竞争力的翻译性能，特别是为英语到哈萨克语翻译确立了新的最先进水平（22.9 > 10.4 BLEU）。我们的代码可在 https://github.com/manestay/EcXTra 获取。
英文摘要：
We propose a two-stage approach for training a single NMT model to translate unseen languages both to and from English. For the first stage, we initialize an encoder-decoder model to pretrained XLM-R and RoBERTa weights, then perform multilingual fine-tuning on parallel data in 40 languages to English. We find this model can generalize to zero-shot translations on unseen languages. For the second stage, we leverage this generalization ability to generate synthetic parallel data from monolingual datasets, then bidirectionally train with successive rounds of back-translation. Our approach, which we EcXTra (English-centric Crosslingual (X) Transfer), is conceptually simple, only using a standard cross-entropy objective throughout. It is also data-driven, sequentially leveraging auxiliary parallel data and monolingual data. We evaluate unsupervised NMT results for 7 low-resource languages, and find that each round of back-translation training further refines bidirectional performance. Our final single EcXTra-trained model achieves competitive translation performance in all translation directions, notably establishing a new state-of-the-art for English-to-Kazakh (22.9 > 10.4 BLEU). Our code is available at https://github.com/manestay/EcXTra .
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
