突厥语多路多语言NMT的评价
Evaluating Multiway Multilingual NMT in the Turkic Languages
摘要：
尽管越来越多的大型和全面的机器翻译（MT）系统，这些方法在各种语言的评估一直受到限制，缺乏高质量的平行语料库，以及与讲这些语言的人的参与。在这项研究中，我们提出了一个国家的最先进的方法来训练和评估MT系统在22种语言从突厥语系，其中大部分是非常不足的探索评估。首先，我们采用了TIL语料库，对训练和评估集进行了一些关键改进。然后，我们训练26双语基线以及多路神经MT（MNMT）模型使用语料库，并进行广泛的分析，使用自动度量以及人类的评价。我们发现MNMT模型在域外测试集中的表现优于几乎所有双语基线，并且在单个对的下游任务上微调模型也会在低资源和高资源情况下导致巨大的性能提升。我们仔细分析了突厥语MT模型的评价标准，也指出了在这个方向上进一步研究的必要性。我们向公众发布语料库分割，测试集以及模型。
英文摘要：
Despite the increasing number of large and comprehensive machine translation (MT) systems, evaluation of these methods in various languages has been restrained by the lack of high-quality parallel corpora as well as engagement with the people that speak these languages. In this study, we present an evaluation of state-of-the-art approaches to training and evaluating MT systems in 22 languages from the Turkic language family, most of which being extremely under-explored. First, we adopt the TIL Corpus with a few key improvements to the training and the evaluation sets. Then, we train 26 bilingual baselines as well as a multi-way neural MT (MNMT) model using the corpus and perform an extensive analysis using automatic metrics as well as human evaluations. We find that the MNMT model outperforms almost all bilingual baselines in the out-of-domain test sets and finetuning the model on a downstream task of a single pair also results in a huge performance boost in both low- and high-resource scenarios. Our attentive analysis of evaluation criteria for MT models in Turkic languages also points to the necessity for further research in this direction. We release the corpus splits, test sets as well as models to the public.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
