arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2210.09805cs.CL
收藏

多领域神经机器翻译的领域特定子网络

Domain Specific Sub-network for Multi-Domain Neural Machine Translation

Microsoft Egypt Development Center(微软埃及开发中心)

Amr Hendy, Mohamed Abdelghaffar, Mohamed Afify, Ahmed Y. Tawfik

摘要:

本文提出了特定域子网(Domain-Specific Sub-Network,DoSS)。它使用通过修剪获得的一组掩码来为每个域定义子网络,并对域数据的子网络参数进行微调。与在每个域上微调整个网络相比,这非常接近并大大减少了参数的数量。此外,提出了一种方法,使每个域的面具唯一的,并大大提高了未见过的域的推广。在我们的德英机器翻译的实验中,所提出的方法比多领域(医疗,技术和宗教)数据的连续训练的强基线高出1.47个BLEU点。此外,继续培训DoSS在新领域(法律)的表现优于多领域(医疗,技术,宗教,法律)基线1.52 BLEU点。

英文摘要:

This paper presents Domain-Specific Sub-network (DoSS). It uses a set of masks obtained through pruning to define a sub-network for each domain and finetunes the sub-network parameters on domain data. This performs very closely and drastically reduces the number of parameters compared to finetuning the whole network on each domain. Also a method to make masks unique per domain is proposed and shown to greatly improve the generalization to unseen domains. In our experiments on German to English machine translation the proposed method outperforms the strong baseline of continue training on multi-domain (medical, tech and religion) data by 1.47 BLEU points. Also continue training DoSS on new domain (legal) outperforms the multi-domain (medical, tech, religion, legal) baseline by 1.52 BLEU points.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑