自适应最近邻机器翻译
Adaptive Nearest Neighbor Machine Translation
摘要:
最近由Khandelwal等人(2020 a)提出的kNN-MT成功地将预训练的神经机器翻译(NMT)模型与令牌级k最近邻(kNN)检索相结合,以提高翻译准确性。然而,传统的kNN算法中使用的kNN MT简单地检索相同数量的最近邻为每个目标令牌,这可能会导致预测错误时,检索的邻居包括噪声。在本文中,我们提出了自适应kNN-MT动态确定每个目标令牌的k的数量。我们通过引入一个轻量级的Meta-k网络来实现这一目标,该网络可以只用很少的训练样本进行有效的训练。在4个标准机器翻译数据集上,实验结果表明,该方法能够有效滤除检索结果中的噪声,且性能明显优于vanilla kNN-MT模型.更值得注意的是,在一个域上学习的Meta-k网络可以直接应用于其他域,并获得一致的改进,说明了我们方法的通用性。我们的实现在https://github.com/zhengxxn/adaptive-knn-mt上开源。
英文摘要:
kNN-MT, recently proposed by Khandelwal et al. (2020a), successfully combines pre-trained neural machine translation (NMT) model with token-level k-nearest-neighbor (kNN) retrieval to improve the translation accuracy. However, the traditional kNN algorithm used in kNN-MT simply retrieves a same number of nearest neighbors for each target token, which may cause prediction errors when the retrieved neighbors include noises. In this paper, we propose Adaptive kNN-MT to dynamically determine the number of k for each target token. We achieve this by introducing a light-weight Meta-k Network, which can be efficiently trained with only a few training samples. On four benchmark machine translation datasets, we demonstrate that the proposed method is able to effectively filter out the noises in retrieval results and significantly outperforms the vanilla kNN-MT model. Even more noteworthy is that the Meta-k Network learned on one domain could be directly applied to other domains and obtain consistent improvements, illustrating the generality of our method. Our implementation is open-sourced at https://github.com/zhengxxn/adaptive-knn-mt.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。