arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2007.10681cs.CL
收藏

具有纠错功能的神经机器翻译

Neural Machine Translation with Error Correction

Nanjing University of Science and Technology(南京理工大学);Microsoft Research Asia(微软亚洲研究院)

Kaitao Song, Xu Tan, Jianfeng Lu

摘要:

神经机器翻译(NMT)在训练过程中生成下一个目标令牌,而在推理过程中生成前一个目标令牌,这会导致训练和推理之间的差异以及错误传播,影响翻译精度。在本文中,我们引入了一个错误校正机制NMT,它纠正了错误的信息,在以前生成的令牌,以更好地预测下一个令牌。具体地说,我们将XLNet的双流自注意引入NMT解码器,其中查询流用于预测下一个令牌,同时内容流用于纠正先前预测令牌的错误信息。我们利用预定采样来模拟训练过程中的预测误差。在三个IWITR翻译数据集和两个WMT翻译数据集上的实验表明,该方法在Transformer基线和预定采样上取得了很好的效果。进一步的实验分析也验证了我们提出的纠错机制对提高翻译质量的有效性。

英文摘要:

Neural machine translation (NMT) generates the next target token given as input the previous ground truth target tokens during training while the previous generated target tokens during inference, which causes discrepancy between training and inference as well as error propagation, and affects the translation accuracy. In this paper, we introduce an error correction mechanism into NMT, which corrects the error information in the previous generated tokens to better predict the next token. Specifically, we introduce two-stream self-attention from XLNet into NMT decoder, where the query stream is used to predict the next token, and meanwhile the content stream is used to correct the error information from the previous predicted tokens. We leverage scheduled sampling to simulate the prediction errors during training. Experiments on three IWSLT translation datasets and two WMT translation datasets demonstrate that our method achieves improvements over Transformer baseline and scheduled sampling. Further experimental analyses also verify the effectiveness of our proposed error correction mechanism to improve the translation quality.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑