arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2104.09810cs.CL
收藏

解决NMT在输入扰动中的漏洞

Addressing the Vulnerability of NMT in Input Perturbations

Institute for Infocomm Research, A*STAR(资讯通信研究院,新加坡科技研究局);The Chinese University of Hong Kong(香港中文大学);Nanyang Technological University(南洋理工大学)

Weiwen Xu, Ai Ti Aw, Yang Ding, Kui Wu, Shafiq Joty

摘要:

神经机器翻译(NMT)在性能上取得了重大突破,但已知其易受输入扰动的影响。由于在训练过程中很难预测真实的输入噪声,因此鲁棒性是系统部署的一个大问题。在本文中,我们提高了NMT模型的鲁棒性,通过上下文增强重建(CER)的方法减少噪音的影响。CER通过两个步骤训练模型抵抗噪声:(1)扰动步骤,用虚构的单词破坏输入序列的自然性;(2)重建步骤,通过生成更好和更鲁棒的上下文表示来防御噪声传播。中文-英文(ZH-EN)和法文-英文(FR-EN)翻译任务的实验结果表明,新闻和社交媒体文本的鲁棒性提高。在社交媒体文本上的进一步微调实验表明,我们的方法可以在更高的位置收敛,并提供更好的适应性。

英文摘要:

Neural Machine Translation (NMT) has achieved significant breakthrough in performance but is known to suffer vulnerability to input perturbations. As real input noise is difficult to predict during training, robustness is a big issue for system deployment. In this paper, we improve the robustness of NMT models by reducing the effect of noisy words through a Context-Enhanced Reconstruction (CER) approach. CER trains the model to resist noise in two steps: (1) perturbation step that breaks the naturalness of input sequence with made-up words; (2) reconstruction step that defends the noise propagation by generating better and more robust contextual representation. Experimental results on Chinese-English (ZH-EN) and French-English (FR-EN) translation tasks demonstrate robustness improvement on both news and social media text. Further fine-tuning experiments on social media text show our approach can converge at a higher position and provide a better adaptation.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑