arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2306.06381cs.CL
收藏

INK:将kNN知识注入近邻机器翻译

INK: Injecting kNN Knowledge in Nearest Neighbor Machine Translation

Nanjing University(南京大学);Shanghai AI Laboratory(上海人工智能实验室);The University of Hong Kong(香港大学)

Wenhao Zhu, Jingjing Xu, Shujian Huang, Lingpeng Kong, Jiajun Chen

AI总结:

针对神经机器翻译表示空间非平滑、kNN-MT推理开销大的问题,提出INK训练框架,通过少量参数调整邻域表示平滑空间并异步更新数据存储,在四个基准上以更低开销取得更优翻译效果。

摘要:

神经机器翻译在众多翻译任务上已取得可观成果,但既往研究表明,神经模型会生成非平滑的表示空间,这会损害其泛化效果。近期,kNN-MT(k近邻机器翻译)提供了一种有效范式,可在推理阶段基于邻域表示平滑预测结果。尽管效果不错,kNN-MT通常存在推理开销大的问题。我们提出了一种有效的训练框架INK,通过少量新增参数调整k近邻的表示,直接平滑表示空间;随后利用这些新增参数异步更新整个表示数据存储,以获取新的kNN知识,该循环持续运行直至收敛。在四个基准数据集上的实验表明,INK方法平均获得1.99 COMET与1.0 BLEU的提升,以0.02倍的内存空间和1.9倍的推理速度优于当前最优的kNN-MT系统。

英文摘要:

Neural machine translation has achieved promising results on many translation tasks. However, previous studies have shown that neural models induce a non-smooth representation space, which harms its generalization results. Recently, kNN-MT has provided an effective paradigm to smooth the prediction based on neighbor representations during inference. Despite promising results, kNN-MT usually requires large inference overhead. We propose an effective training framework INK to directly smooth the representation space via adjusting representations of kNN neighbors with a small number of new parameters. The new parameters are then used to refresh the whole representation datastore to get new kNN knowledge asynchronously. This loop keeps running until convergence. Experiments on four benchmark datasets show that \method achieves average gains of 1.99 COMET and 1.0 BLEU, outperforming the state-of-the-art kNN-MT system with 0.02x memory space and 1.9x inference speedup.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑