基于相似度衰减的对比token学习用于机器翻译中的重复抑制
Contrastive Token Learning with Similarity Decay for Repetition Suppression in Machine Translation
AI总结:
本文提出CTSD算法,通过信息熵分析重复成因,利用注意力权重与token距离动态抑制重复,并在自建电商标题数据集及阿里巴巴国际站八个站点全量上线中验证了其优于现有方法的翻译精确度、泛化性和商业转化效果。
摘要:
对于跨语言对话与贸易而言,神经机器翻译(NMT)至关重要,但其生成内容中的单调性和重复问题始终构成持续挑战。传统解决方案依赖于惩罚文本冗余或token重复出现,即便在大语言模型(LLMs)出现之后,对于长篇文章和本身具有固有冗余的电商描述,其效果仍然有限。本文从信息熵的视角探究文本重复的深层原因,将该现象归因于输入文本内部升高的不确定性。为此,本文提出一种名为基于相似度衰减的对比token学习(CTSD)的新算法,该算法依据不同的注意力权重和token间距离动态调节对token的抑制。此外,本文还编制并发布了一个由在线真实商品标题文本组成的电商数据集,该数据集易受幻觉翻译影响,用于对该算法进行基准测试。大量评测表明,CTSD在精确度和泛化性方面显著优于现有方法。额外的在线A/B测试进一步印证了其实用价值,显示出用户参与度和转化率的明显提升。值得注意的是,该方法已在阿里巴巴国际站(alibaba.com)——全球最大的B2B电商平台——的八个多语言站点上全量上线。
英文摘要:
For crosslingual conversation and trade, Neural Machine Translation (NMT) is pivotal yet faces persistent challenges with monotony and repetition in generated content. Traditional solutions that rely on penalizing text redundancy or token reoccurrence have shown limited efficacy, particularly for lengthy article and e-commerce descriptions with inherent redundancy, even with the advent of Large Language Models (LLMs). This paper investigates the underlying causes of textual repetition through the lens of information entropy, attributing the phenomenon to the elevated uncertainty within the input text. To address this, a novel algorithm named Contrastive Token Learning with Similarity Decay (CTSD) is introduced, which modulates the suppression of tokens dynamically, informed by varying attention weights and inter-token distances. Furthermore, an e-commerce dataset comprised of title texts of online real items is compiled and released susceptible to hallucination translations to benchmark the algorithm. Extensive evaluations demonstrate that CTSD significantly outperforms existing approaches in precision and generalizability. Additional online A/B testing underscores its practical value, showing marked improvements in user engagement and conversion. Notably, this method has been implemented with full traffic on eight multilingual sites of alibaba.com, the largest B2B e-commerce platform in the world.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。