arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1908.09324cs.CLcs.LG
收藏

基于语言聚类的多语言神经机器翻译

Multilingual Neural Machine Translation with Language Clustering

University of Science and Technology of China(中国科学技术大学);Peking University(北京大学);Microsoft Research(微软研究院)

Xu Tan, Jiale Chen, Di He, Yingce Xia, Tao Qin, Tie-Yan Liu

摘要:

多语言神经机器翻译(NMT)使用单个模型翻译多种语言,由于其在简化训练过程、降低在线维护成本以及增强低资源和zero-shot翻译方面的优势,具有重要的实际意义。考虑到世界上有数千种语言,其中一些语言非常不同,在单个模型中处理所有语言或为每个语言对使用单独的模型是非常繁重的。因此,给定一个固定的资源预算,例如,模型的数量,如何确定哪些语言应该支持的一个模型是至关重要的多语言NMT,这,不幸的是,已被忽略了以前的工作。在这项工作中,我们开发了一个框架,将语言聚类到不同的组中,并为每个聚类训练一个多语言模型。我们研究了两种语言聚类方法:(1)使用先验知识,根据语言家族对语言进行聚类;(2)使用语言嵌入,用嵌入向量表示每种语言,并在嵌入空间中对它们进行聚类。特别地,我们通过训练通用神经机器翻译模型来获得所有语言的嵌入向量。我们对23种语言的实验表明,第一种聚类方法简单易懂,但翻译准确率不高,而第二种方法充分捕捉了语言之间的关系,与基线方法相比,几乎所有语言的翻译准确率都有所提高

英文摘要:

Multilingual neural machine translation (NMT), which translates multiple languages using a single model, is of great practical importance due to its advantages in simplifying the training process, reducing online maintenance costs, and enhancing low-resource and zero-shot translation. Given there are thousands of languages in the world and some of them are very different, it is extremely burdensome to handle them all in a single model or use a separate model for each language pair. Therefore, given a fixed resource budget, e.g., the number of models, how to determine which languages should be supported by one model is critical to multilingual NMT, which, unfortunately, has been ignored by previous work. In this work, we develop a framework that clusters languages into different groups and trains one multilingual model for each cluster. We study two methods for language clustering: (1) using prior knowledge, where we cluster languages according to language family, and (2) using language embedding, in which we represent each language by an embedding vector and cluster them in the embedding space. In particular, we obtain the embedding vectors of all the languages by training a universal neural machine translation model. Our experiments on 23 languages show that the first clustering method is simple and easy to understand but leading to suboptimal translation accuracy, while the second method sufficiently captures the relationship among languages well and improves the translation accuracy for almost all the languages over baseline methods

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑