arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2210.08808cs.CL
收藏

鲁棒k-近邻机器翻译

Towards Robust k-Nearest-Neighbor Machine Translation

Xiamen University(厦门大学);Tencent Inc(腾讯公司);Dalian University of Technology(大连理工大学);Pengcheng Laboratory(鹏城实验室)

Hui Jiang, Ziyao Lu, Fandong Meng, Chulun Zhou, Jie Zhou, Degen Huang, Jinsong Su

摘要:

最近邻机器翻译(kNN-MT)是近年来NMT的一个重要研究方向。它的主要思想是从一个额外的索引中检索有用的键值对来修改翻译,而不需要更新NMT模型。然而,底层检索到的噪声对将显著恶化模型性能。在本文中,我们进行了初步的研究,发现这一问题的结果没有充分利用NMT模型的预测。为了减轻噪声的影响,我们提出了一种具有鲁棒训练的置信度增强的kNN-MT模型。具体而言,我们引入NMT置信度来细化kNN-MT的两个重要组成部分的建模:kNN分布和插值权重。同时,我们注入两种类型的扰动到检索对鲁棒训练。在四个基准数据集上的实验结果表明,我们的模型不仅取得了显着的改进,目前kNN-MT模型,但也表现出更好的鲁棒性。我们的代码可在https://github.com/DeepLearnXMU/Robust-knn-mt上获得。

英文摘要:

k-Nearest-Neighbor Machine Translation (kNN-MT) becomes an important research direction of NMT in recent years. Its main idea is to retrieve useful key-value pairs from an additional datastore to modify translations without updating the NMT model. However, the underlying retrieved noisy pairs will dramatically deteriorate the model performance. In this paper, we conduct a preliminary study and find that this problem results from not fully exploiting the prediction of the NMT model. To alleviate the impact of noise, we propose a confidence-enhanced kNN-MT model with robust training. Concretely, we introduce the NMT confidence to refine the modeling of two important components of kNN-MT: kNN distribution and the interpolation weight. Meanwhile we inject two types of perturbations into the retrieved pairs for robust training. Experimental results on four benchmark datasets demonstrate that our model not only achieves significant improvements over current kNN-MT models, but also exhibits better robustness. Our code is available at https://github.com/DeepLearnXMU/Robust-knn-mt.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑