arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2109.04593cs.CLcs.LG
收藏

大规模突厥语机器翻译研究

A Large-Scale Study of Machine Translation in the Turkic Languages

University of South Florida(南佛罗里达大学);NYU(纽约大学);Indiana University(印第安纳大学);EPFL(洛桑联邦理工学院);University of Helsinki(赫尔辛基大学);Namangan State University(纳曼干州立大学);Google Research(谷歌研究院)
Jamshidbek Mirzakhalov, Anoop Babu, Duygu Ataman, Sherzod Kariev, Francis Tyers, Otabek Abduraufov, Mammad Hajili, Sardana Ivanova, Abror Khaytbaev, Antonio Lav… 展开作者

Jamshidbek Mirzakhalov, Anoop Babu, Duygu Ataman, Sherzod Kariev, Francis Tyers, Otabek Abduraufov, Mammad Hajili, Sardana Ivanova, Abror Khaytbaev, Antonio Laverghetta, Behzodbek Moydinboyev, Esra Onal, Shaxnoza Pulatova, Ahsan Wahab, Orhan Firat, Sriram Chellappan

摘要:

神经机器翻译(NMT)的最新进展已经将机器翻译系统的质量推到了它们被广泛采用以构建有竞争力的系统的程度。然而,仍然有大量的语言尚未获得NMT的好处。在本文中,我们提供了第一个大规模的案例研究的实际应用中的MT的突厥语系,以实现收益的NMT的突厥语在高资源的极端低资源的情况下。除了提出一个广泛的分析,确定建立有竞争力的系统,以改善数据稀缺的瓶颈,我们的研究有几个关键的贡献,包括,i)一个大型的平行语料库,涵盖22个突厥语语言组成的共同公共数据集,结合新的数据集约200万个平行句子,ii)双语基线为26对语言,iii)三个不同翻译领域的新型高质量测试集和iv)人类评估分数。所有模型、脚本和数据都将向公众发布。

英文摘要:

Recent advances in neural machine translation (NMT) have pushed the quality of machine translation systems to the point where they are becoming widely adopted to build competitive systems. However, there is still a large number of languages that are yet to reap the benefits of NMT. In this paper, we provide the first large-scale case study of the practical application of MT in the Turkic language family in order to realize the gains of NMT for Turkic languages under high-resource to extremely low-resource scenarios. In addition to presenting an extensive analysis that identifies the bottlenecks towards building competitive systems to ameliorate data scarcity, our study has several key contributions, including, i) a large parallel corpus covering 22 Turkic languages consisting of common public datasets in combination with new datasets of approximately 2 million parallel sentences, ii) bilingual baselines for 26 language pairs, iii) novel high-quality test sets in three different translation domains and iv) human evaluation scores. All models, scripts, and data will be released to the public.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑