arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2002.06546cs.CL

Neural Machine Translation with Joint Representation

Northeastern University(东北大学);NiuTrans Co., Ltd.(小牛翻译股份有限公司);Institute of Psychology, CAS(中国科学院心理研究所)

Yanyang Li, Qiang Wang, Tong Xiao, Tongran Liu, Jingbo Zhu

英文摘要:

Though early successes of Statistical Machine Translation (SMT) systems are attributed in part to the explicit modelling of the interaction between any two source and target units, e.g., alignment, the recent Neural Machine Translation (NMT) systems resort to the attention which partially encodes the interaction for efficiency. In this paper, we employ Joint Representation that fully accounts for each possible interaction. We sidestep the inefficiency issue by refining representations with the proposed efficient attention operation. The resulting Reformer models offer a new Sequence-to- Sequence modelling paradigm besides the Encoder-Decoder framework and outperform the Transformer baseline in either the small scale IWSLT14 German-English, English-German and IWSLT15 Vietnamese-English or the large scale NIST12 Chinese-English translation tasks by about 1 BLEU point.We also propose a systematic model scaling approach, allowing the Reformer model to beat the state-of-the-art Transformer in IWSLT14 German-English and NIST12 Chinese-English with about 50% fewer parameters. The code is publicly available at https://github.com/lyy1994/reformer.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑