arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1912.07239cs.CL
收藏

神经机器翻译的迭代双领域自适应

Iterative Dual Domain Adaptation for Neural Machine Translation

Xiamen University(厦门大学);Tsinghua University(清华大学);University of Illinois at Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);Institute of Software, Chinese Academy of Sciences(中国科学院软件研究所);University of Rochester(罗切斯特大学)

Jiali Zeng, Yang Liu, Jinsong Su, Yubin Ge, Yaojie Lu, Yongjing Yin, Jiebo Luo

摘要:

以往神经机器翻译领域自适应的研究主要集中在将领域外的翻译知识一次性转换到领域内的机器翻译模型上。在本文中,我们认为,这样的策略未能充分提取领域共享的翻译知识,和重复利用不同领域的语料库,可以导致更好地提炼领域共享的翻译知识。为此,我们提出了一个迭代的双重领域自适应框架的NMT。具体而言,首先分别使用域内和域外NMT模型的训练语料对它们进行预训练,然后基于知识蒸馏迭代地进行双向翻译知识转移(域内到域外,反之亦然),直到域内NMT模型收敛。此外,我们将所提出的框架扩展到多个域外训练语料库的场景,其中上述转移是在域内和每个域外NMT模型之间按其域相似性的升序顺序依次执行的。在汉英和英德翻译任务上的实验结果证明了该框架的有效性。

英文摘要:

Previous studies on the domain adaptation for neural machine translation (NMT) mainly focus on the one-pass transferring out-of-domain translation knowledge to in-domain NMT model. In this paper, we argue that such a strategy fails to fully extract the domain-shared translation knowledge, and repeatedly utilizing corpora of different domains can lead to better distillation of domain-shared translation knowledge. To this end, we propose an iterative dual domain adaptation framework for NMT. Specifically, we first pre-train in-domain and out-of-domain NMT models using their own training corpora respectively, and then iteratively perform bidirectional translation knowledge transfer (from in-domain to out-of-domain and then vice versa) based on knowledge distillation until the in-domain NMT model convergences. Furthermore, we extend the proposed framework to the scenario of multiple out-of-domain training corpora, where the above-mentioned transfer is performed sequentially between the in-domain and each out-of-domain NMT models in the ascending order of their domain similarities. Empirical results on Chinese-English and English-German translation tasks demonstrate the effectiveness of our framework.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑