多语言翻译中干扰的原因与对策
Causes and Cures for Interference in Multilingual Translation
AI总结：
本研究系统探究了多语言机器翻译中干扰现象的主要成因，发现干扰主要由模型规模、数据规模及各语言对比例决定，并提出通过使用十亿参数以下标准transformer配置及调节采样温度来有效缓解干扰、促进协同并提升整体性能。
摘要：
多语言机器翻译模型能够从不同语言对之间的协同中获益，但也会受到干扰。尽管旨在消除干扰的复杂方法日益增多，但我们对干扰这一现象的理解仍然有限。本研究识别了导致多语言机器翻译中干扰的主要因素。通过系统性实验，我们发现干扰（或协同）主要由模型规模、数据规模以及各语言对在总数据集中所占的比例决定。我们观察到，当模型规模相对于可用训练数据非常小时，才会出现显著的干扰；而使用参数量少于十亿的标准 transformer 配置，则能在很大程度上缓解干扰并促进协同。此外，我们表明，调节采样温度以控制数据中各语言对的比例，是有效平衡低资源与高资源语言对之间干扰程度的关键，并能够带来整体上更优的性能。
英文摘要：
Multilingual machine translation models can benefit from synergy between different language pairs, but also suffer from interference. While there is a growing number of sophisticated methods that aim to eliminate interference, our understanding of interference as a phenomenon is still limited. This work identifies the main factors that contribute to interference in multilingual machine translation. Through systematic experimentation, we find that interference (or synergy) are primarily determined by model size, data size, and the proportion of each language pair within the total dataset. We observe that substantial interference occurs mainly when the model is very small with respect to the available training data, and that using standard transformer configurations with less than one billion parameters largely alleviates interference and promotes synergy. Moreover, we show that tuning the sampling temperature to control the proportion of each language pair in the data is key to balancing the amount of interference between low and high resource language pairs effectively, and can lead to superior performance overall.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
