用于神经机器翻译预训练的基于词义的语码转换
Code-Switching with Word Senses for Pretraining in Neural Machine Translation
AI总结：
针对NMT预训练中忽略词义导致偏差的问题，提出WSP-NMT方法，利用知识库的词义信息进行端到端预训练，显著提升翻译质量并在DiBiMT基准上取得细粒度准确率提升。
摘要：
词汇歧义是神经机器翻译（NMT）中一个重大且普遍的挑战，许多最先进的（SOTA）NMT系统都难以处理多义词。在去噪合成“语码转换”文本的NMT预训练范式中情况同样如此，词义在加噪阶段被忽略，导致预训练数据中产生有害的词义偏差，这些偏差随后被生成的模型所继承。在这项工作中，我们引入了用于神经机器翻译的词义预训练（WSP-NMT）——一种端到端的方法，利用来自知识库的特定词义信息来预训练多语言NMT模型。我们的实验表明，整体翻译质量有显著提升。然后，我们展示了我们的方法在扩展到各种具有挑战性的数据和资源稀缺场景时的鲁棒性，最后报告了在DiBiMT消歧基准上的细粒度准确率提升。我们的研究为在NMT的多语言预训练中整合词义信息和结构化知识的优点与挑战提供了有趣且新颖的见解。
英文摘要：
Lexical ambiguity is a significant and pervasive challenge in Neural Machine Translation (NMT), with many state-of-the-art (SOTA) NMT systems struggling to handle polysemous words (Campolungo et al., 2022). The same holds for the NMT pretraining paradigm of denoising synthetic "code-switched" text (Pan et al., 2021; Iyer et al., 2023), where word senses are ignored in the noising stage -- leading to harmful sense biases in the pretraining data that are subsequently inherited by the resulting models. In this work, we introduce Word Sense Pretraining for Neural Machine Translation (WSP-NMT) - an end-to-end approach for pretraining multilingual NMT models leveraging word sense-specific information from Knowledge Bases. Our experiments show significant improvements in overall translation quality. Then, we show the robustness of our approach to scale to various challenging data and resource-scarce scenarios and, finally, report fine-grained accuracy improvements on the DiBiMT disambiguation benchmark. Our studies yield interesting and novel insights into the merits and challenges of integrating word sense information and structured knowledge in multilingual pretraining for NMT.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
