arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2107.06569cs.CL
收藏

多语言神经机器翻译中基于重要度的神经元分配

Importance-based Neuron Allocation for Multilingual Neural Machine Translation

Institute of Computing Technology, Chinese Academy of Sciences(中国科学院计算技术研究所);University of Chinese Academy of Sciences(中国科学院大学);Beijing Language and Culture University(北京语言大学)

Wanying Xie, Yang Feng, Shuhao Gu, Dong Yu

摘要:

基于单一模型的多语言神经机器翻译因其能够处理多种语言的能力而受到广泛关注。然而,目前的多语种翻译范式往往使模型倾向于保留通用知识,而忽略了语言特有的知识。以往的一些工作试图通过在模型中添加各种语言特定的模块来解决这个问题,但它们存在参数爆炸问题,需要专门的手工设计。为了解决这些问题,我们建议根据模型神经元在不同语言中的重要性将其分为通用和特定语言部分。一般部分负责保存一般知识并参与所有语言的翻译,而语言特定部分负责保存语言特定知识并参与某些特定语言的翻译。实验结果表明,该方法具有较好的通用性和有效性.

英文摘要:

Multilingual neural machine translation with a single model has drawn much attention due to its capability to deal with multiple languages. However, the current multilingual translation paradigm often makes the model tend to preserve the general knowledge, but ignore the language-specific knowledge. Some previous works try to solve this problem by adding various kinds of language-specific modules to the model, but they suffer from the parameter explosion problem and require specialized manual design. To solve these problems, we propose to divide the model neurons into general and language-specific parts based on their importance across languages. The general part is responsible for preserving the general knowledge and participating in the translation of all the languages, while the language-specific part is responsible for preserving the language-specific knowledge and participating in the translation of some specific languages. Experimental results on several language pairs, covering IWSLT and Europarl corpus datasets, demonstrate the effectiveness and universality of the proposed method.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑