神经机器翻译小批量生成策略的实证研究
An Empirical Study of Mini-Batch Creation Strategies for Neural Machine Translation
摘要：
神经机器翻译（NMT）模型的训练通常使用小批量来提高效率。在小批量训练过程中，为了有效计算，需要填充小批量中的较短句子，使其长度等于其中的最长句子。以前的工作已经注意到，在进行小批量处理之前，根据句子长度对语料库进行排序，减少了填充量，提高了处理速度。然而，尽管小批量创建是NMT训练中的一个重要步骤，但广泛使用的NMT工具包实现了不同的策略，这些策略尚未经过经验验证或比较。本文研究了小批量创建策略，并在两个不同的数据集上进行了实验。我们的研究结果表明，小批量创建策略的选择对NMT训练有很大的影响，与简单的洗牌相比，一些基于长度的排序策略并不总是有效。
英文摘要：
Training of neural machine translation (NMT) models usually uses mini-batches for efficiency purposes. During the mini-batched training process, it is necessary to pad shorter sentences in a mini-batch to be equal in length to the longest sentence therein for efficient computation. Previous work has noted that sorting the corpus based on the sentence length before making mini-batches reduces the amount of padding and increases the processing speed. However, despite the fact that mini-batch creation is an essential step in NMT training, widely used NMT toolkits implement disparate strategies for doing so, which have not been empirically validated or compared. This work investigates mini-batch creation strategies with experiments over two different datasets. Our results suggest that the choice of a mini-batch creation strategy has a large effect on NMT training and some length-based sorting strategies do not always work well compared with simple shuffling.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
