专注于目标词汇:机器翻译的掩蔽标签平滑
Focus on the Target's Vocabulary: Masked Label Smoothing for Machine Translation
摘要:
标签平滑和词汇共享是神经机器翻译模型中广泛使用的两种技术。然而,我们认为,简单地应用这两种技术可能是冲突的,甚至导致次优性能。在分配平滑概率时,原始标签平滑会将源端词中不可能出现在目标语言中的词与目标语言中的词等同对待,这会使翻译模型产生偏差。为了解决这个问题,我们提出了屏蔽标签平滑(MLS),一个新的机制,掩盖了软标签的源端字的概率为零。简单而有效,MLS设法更好地集成标签平滑与词汇共享。我们广泛的实验表明,MLS在不同的数据集上,包括双语和多语种翻译,无论是翻译质量还是模型的校准,都比原始标签平滑得到了持续的改进。我们的代码发布于https://github.com/PKUnlp-icler/MLS
英文摘要:
Label smoothing and vocabulary sharing are two widely used techniques in neural machine translation models. However, we argue that simply applying both techniques can be conflicting and even leads to sub-optimal performance. When allocating smoothed probability, original label smoothing treats the source-side words that would never appear in the target language equally to the real target-side words, which could bias the translation model. To address this issue, we propose Masked Label Smoothing (MLS), a new mechanism that masks the soft label probability of source-side words to zero. Simple yet effective, MLS manages to better integrate label smoothing with vocabulary sharing. Our extensive experiments show that MLS consistently yields improvement over original label smoothing on different datasets, including bilingual and multilingual translation from both translation quality and model's calibration. Our code is released at https://github.com/PKUnlp-icler/MLS
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。