arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2204.06175cs.CL
收藏

一种高效的k-近邻机器翻译算法

Efficient Cluster-Based k-Nearest-Neighbor Machine Translation

Tianjin University(天津大学)

Dexin Wang, Kai Fan, Boxing Chen, Deyi Xiong

摘要:

最近提出的k近邻机器翻译(kNN-MT)是神经机器翻译(NMT)领域自适应的一种非参数解决方案。它的目的是减轻先进的机器翻译系统在翻译领域外的句子的性能下降,通过协调与一个额外的令牌级的基于特征的检索模块构建从域内数据。以前的研究已经证明,非参数NMT甚至优于对域外数据进行微调的模型。尽管取得了这一成功,但kNN检索是以高延迟为代价的,特别是对于大型数据存储。为了使其实用,在本文中,我们探索了一个更有效的kNN-MT,并提出使用聚类来提高检索效率。具体地说,我们首先提出了一个基于聚类的紧凑网络的对比学习方式的特征约简压缩到90+%的低维向量的上下文特征。然后,我们提出了一个基于集群的修剪解决方案,过滤掉10%-40%的冗余节点在大型数据存储,同时保持翻译质量。我们提出的方法实现了更好的或相当的性能,同时减少了57%的推理延迟对先进的非参数MT模型在几个机器翻译基准。实验结果表明,所提出的方法保持了原始网络的最有用的信息,紧凑网络在未知领域表现出良好的泛化能力。

英文摘要:

k-Nearest-Neighbor Machine Translation (kNN-MT) has been recently proposed as a non-parametric solution for domain adaptation in neural machine translation (NMT). It aims to alleviate the performance degradation of advanced MT systems in translating out-of-domain sentences by coordinating with an additional token-level feature-based retrieval module constructed from in-domain data. Previous studies have already demonstrated that non-parametric NMT is even superior to models fine-tuned on out-of-domain data. In spite of this success, kNN retrieval is at the expense of high latency, in particular for large datastores. To make it practical, in this paper, we explore a more efficient kNN-MT and propose to use clustering to improve the retrieval efficiency. Concretely, we first propose a cluster-based Compact Network for feature reduction in a contrastive learning manner to compress context features into 90+% lower dimensional vectors. We then suggest a cluster-based pruning solution to filter out 10%-40% redundant nodes in large datastores while retaining translation quality. Our proposed methods achieve better or comparable performance while reducing up to 57% inference latency against the advanced non-parametric MT model on several machine translation benchmarks. Experimental results indicate that the proposed methods maintain the most useful information of the original datastore and the Compact Network shows good generalization on unseen domains.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑