迈向多语言机器翻译中更高的帕累托前沿
Towards Higher Pareto Frontier in Multilingual Machine Translation
AI总结：
本文提出帕累托互蒸馏框架 Pareto-MD，通过协同训练两个偏向不同语言的帕累托最优解并利用知识蒸馏互相学习，在 WMT 和 TED 数据集上显著推进多语言机器翻译的帕累托前沿，BLEU 最高提升 2.46。
摘要：
近年来，多语言神经机器翻译取得了显著进展。然而，多语言语料库的长尾分布给帕累托优化带来了挑战，即优化某些语言可能会以降低其他语言性能为代价。现有的平衡训练策略等价于一系列帕累托最优解，它们在帕累托前沿上进行权衡取舍。在这项工作中，我们提出了一种新的训练框架——帕累托互蒸馏（Pareto-MD），旨在将帕累托前沿向外推进，而不是进行权衡取舍。具体而言，Pareto-MD 协同训练两个偏向不同语言的帕累托最优解，并通过知识蒸馏使它们互相学习各自的优势。此外，我们引入了一种新颖的策略，以增强帕累托最优解之间的通信，并扩大我们方法的适用范围。在广泛使用的 WMT 和 TED 数据集上的实验结果表明，我们的方法显著推进了帕累托前沿，并在 BLEU 指标上比基线最多高出 +2.46。
英文摘要：
Multilingual neural machine translation has witnessed remarkable progress in recent years. However, the long-tailed distribution of multilingual corpora poses a challenge of Pareto optimization, i.e., optimizing for some languages may come at the cost of degrading the performance of others. Existing balancing training strategies are equivalent to a series of Pareto optimal solutions, which trade off on a Pareto frontier. In this work, we propose a new training framework, Pareto Mutual Distillation (Pareto-MD), towards pushing the Pareto frontier outwards rather than making trade-offs. Specifically, Pareto-MD collaboratively trains two Pareto optimal solutions that favor different languages and allows them to learn from the strengths of each other via knowledge distillation. Furthermore, we introduce a novel strategy to enable stronger communication between Pareto optimal solutions and broaden the applicability of our approach. Experimental results on the widely-used WMT and TED datasets show that our method significantly pushes the Pareto frontier and outperforms baselines by up to +2.46 BLEU.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
