arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2005.01483cs.CL
收藏

在神经机器翻译训练目标中使用上下文

Using Context in Neural Machine Translation Training Objectives

University of Cambridge(剑桥大学);Google(谷歌)

Danielle Saunders, Felix Stahlberg, Bill Byrne

摘要:

我们提出了神经机器翻译(NMT)训练,使用文档级指标和批量级文档。以前的NMT训练序列目标方法只关注句子BLEU等句子级指标,这些指标与所需的评估指标(通常是文档BLEU)不对应。同时,文档级NMT训练的研究主要集中在数据或模型的体系结构上,而不是训练过程。我们发现,这些研究的每一条线都有一个明确的空间,并建议将它们与一个允许在NMT训练目标中使用文档级评估指标的方案合并。我们首先从句子样本中抽取伪文档。然后,我们用蒙特卡洛采样近似预期的文档BLEU梯度,用作最小风险训练(MRT)中的成本函数。这种两级采样过程使NMT的性能优于序列MRT和最大似然训练。我们证明,培训是更强大的文档级指标比序列指标。我们进一步展示了NMT与TER和语法纠错(GEC)使用GLEU,这两个指标在文档级别的评估的改进。

英文摘要:

We present Neural Machine Translation (NMT) training using document-level metrics with batch-level documents. Previous sequence-objective approaches to NMT training focus exclusively on sentence-level metrics like sentence BLEU which do not correspond to the desired evaluation metric, typically document BLEU. Meanwhile research into document-level NMT training focuses on data or model architecture rather than training procedure. We find that each of these lines of research has a clear space in it for the other, and propose merging them with a scheme that allows a document-level evaluation metric to be used in the NMT training objective. We first sample pseudo-documents from sentence samples. We then approximate the expected document BLEU gradient with Monte Carlo sampling for use as a cost function in Minimum Risk Training (MRT). This two-level sampling procedure gives NMT performance gains over sequence MRT and maximum-likelihood training. We demonstrate that training is more robust for document-level metrics than with sequence metrics. We further demonstrate improvements on NMT with TER and Grammatical Error Correction (GEC) using GLEU, both metrics used at the document level for evaluations.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑