arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2410.04147cs.CL
收藏

模型权重变化能否作为自步多语言神经机器翻译的判据?

Can the Variation of Model Weights be used as a Criterion for Self-Paced Multilingual NMT?

HEIG-VD / HES-SO;EPFL(洛桑联邦理工学院);Armasuisse W+T

Àlex R. Atrio, Alexis Allemann, Ljiljana Dolamic, Andrei Popescu-Belis

AI总结:

针对数据稀缺场景下的多对一神经机器翻译训练,提出基于Transformer层间平滑KL散度判断权重变化、动态切换小批量语言的自步算法,实验显示其效果优于交替单语言批次但弱于混洗批次。

摘要:

在训练数据稀缺时,多对一神经机器翻译系统的表现优于一对一系统。本文设计并测试了一种在训练此类系统时选择小批量数据语言的新算法。当模型权重未发生显著变化时,该算法会切换小批量数据的语言,权重变化程度通过Transformer网络所有层之间的平滑KL散度衡量。在翻译质量(采用BLEU和COMET指标)和收敛速度方面,该算法的表现优于交替单语言批次策略,但不及混洗批次策略。

英文摘要:

Many-to-one neural machine translation systems improve over one-to-one systems when training data is scarce. In this paper, we design and test a novel algorithm for selecting the language of minibatches when training such systems. The algorithm changes the language of the minibatch when the weights of the model do not evolve significantly, as measured by the smoothed KL divergence between all layers of the Transformer network. This algorithm outperforms the use of alternating monolingual batches, but not the use of shuffled batches, in terms of translation quality (measured with BLEU and COMET) and convergence speed.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑