arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2201.05700cs.CLcs.AI
收藏

低资源神经机器翻译中的成本效益训练

Cost-Effective Training in Low-Resource Neural Machine Translation

Maastricht University(马斯特里赫特大学)

Sai Koneru, Danni Liu, Jan Niehues

摘要:

虽然在神经机器翻译(NMT)中探索了主动学习(AL)技术,但只有少数作品专注于解决低注释预算,其中可以翻译的句子数量有限。这种情况尤其具有挑战性,并且可能发生在濒危语言中,这些语言很少有人类注释者或具有标记大量数据的成本限制。虽然AL被证明是有帮助的大预算,这是不够的,以建立高质量的翻译系统,在这些低资源的条件。在这项工作中,我们提出了一个具有成本效益的训练过程,利用少量的注释句子和字典条目来提高NMT模型的性能。我们的方法利用单语数据与自我监督的目标和一个小规模的,廉价的字典进行额外的监督,以初始化NMT模型之前,应用AL。我们表明,使用这些知识源的组合来改进模型是必不可少的利用AL策略,并增加收益在低资源条件下。我们还提出了一种新的AL策略的启发领域自适应的NMT,并表明它是有效的低预算。我们提出了一种新的混合数据驱动的方法,该方法对与标记数据不同且与未标记数据最相似的句子进行采样。最后,我们表明,初始化的NMT模型,并进一步使用我们的AL策略可以实现收益高达$13$ BLEU相比,传统的AL方法。

英文摘要:

While Active Learning (AL) techniques are explored in Neural Machine Translation (NMT), only a few works focus on tackling low annotation budgets where a limited number of sentences can get translated. Such situations are especially challenging and can occur for endangered languages with few human annotators or having cost constraints to label large amounts of data. Although AL is shown to be helpful with large budgets, it is not enough to build high-quality translation systems in these low-resource conditions. In this work, we propose a cost-effective training procedure to increase the performance of NMT models utilizing a small number of annotated sentences and dictionary entries. Our method leverages monolingual data with self-supervised objectives and a small-scale, inexpensive dictionary for additional supervision to initialize the NMT model before applying AL. We show that improving the model using a combination of these knowledge sources is essential to exploit AL strategies and increase gains in low-resource conditions. We also present a novel AL strategy inspired by domain adaptation for NMT and show that it is effective for low budgets. We propose a new hybrid data-driven approach, which samples sentences that are diverse from the labelled data and also most similar to unlabelled data. Finally, we show that initializing the NMT model and further using our AL strategy can achieve gains of up to $13$ BLEU compared to conventional AL methods.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑