基于任务级混合专家模型的直接神经机器翻译
Direct Neural Machine Translation with Task-level Mixture of Experts models
AI总结：
针对非英语语言对平行数据稀缺导致直接神经机器翻译性能受限的问题，本研究探索任务级混合专家（Task-level MoE）模型的适用性，提出高性能训练评估配置，在多数语言对和翻译方向上超越双语与枢轴翻译模型。
摘要：
直接神经机器翻译（direct NMT）是一类在两种非英语语言之间翻译文本的NMT系统。由于非英语语言对之间的平行语料稀缺，直接NMT系统往往面临局限性。已有多种方法被提出以解决这一局限，例如多语言NMT和枢轴NMT（通过英语在两种语言之间进行翻译）。任务级混合专家模型（Task-level MoE）是基于Transformer的模型的一种推理高效变体，已在大量语言对的NMT任务中展现出可观的性能。在Task-level MoE中，不同语言组可采用不同的路由策略，以优化跨语言学习效果和推理速度。本研究考察了Task-level MoE在直接NMT中的适用性，并提出了一系列高性能的训练与评估配置。通过这些配置，基于Task-level MoE的直接NMT系统在大量低资源和高资源直接语言对以及翻译方向上，性能优于双语模型和基于枢轴的模型。我们的含16个专家的Task-level MoE在7个语言对上的表现优于双语NMT、枢轴NMT模型，而基于枢轴的模型在9个语言对和翻译方向上的表现仍然更优。
英文摘要：
Direct neural machine translation (direct NMT) is a type of NMT system that translates text between two non-English languages. Direct NMT systems often face limitations due to the scarcity of parallel data between non-English language pairs. Several approaches have been proposed to address this limitation, such as multilingual NMT and pivot NMT (translation between two languages via English). Task-level Mixture of expert models (Task-level MoE), an inference-efficient variation of Transformer-based models, has shown promising NMT performance for a large number of language pairs. In Task-level MoE, different language groups can use different routing strategies to optimize cross-lingual learning and inference speed. In this work, we examine Task-level MoE's applicability in direct NMT and propose a series of high-performing training and evaluation configurations, through which Task-level MoE-based direct NMT systems outperform bilingual and pivot-based models for a large number of low and high-resource direct pairs, and translation directions. Our Task-level MoE with 16 experts outperforms bilingual NMT, Pivot NMT models for 7 language pairs, while pivot-based models still performed better in 9 pairs and directions.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
