用于跨语言迁移的机器创建通用语言
Machine-Created Universal Language for Cross-lingual Transfer
AI总结：
针对跨语言迁移中translate-test性能不足、多语言预训练可解释性差且难适配词级任务的问题，提出机器创建通用语言MUL作为中间语言，兼顾迁移性能优于多语言预训练且可解释性强。
摘要：
解决跨语言迁移主要有两种方法：多语言预训练，它隐式对齐不同语言的隐藏表示；以及translate-test（翻译-测试），它显式将不同语言翻译成英语等中间语言。与多语言预训练相比，translate-test的可解释性更好，但性能低于多语言预训练（Conneau和Lample，2019；Conneau等人，2020），且由于翻译会改变词序，难以应对词级任务。为此，我们提出一种新的机器创建通用语言（Machine-created Universal Language，MUL）作为替代中间语言。MUL由一组构成通用词汇表的离散符号，以及一个用于将多种自然语言转换为MUL的自然语言到MUL的翻译器组成。MUL将不同语言的共享概念统一为单个通用词，增强了跨语言迁移效果。此外，MUL保留了特定语言的词汇和词序，使模型能轻松应用于词级任务。我们的实验表明，翻译成MUL相比多语言预训练能带来性能提升，分析也显示MUL具备很强的可解释性。代码位于：https://github.com/microsoft/Unicoder/tree/master/MCUL。
英文摘要：
There are two primary approaches to addressing cross-lingual transfer: multilingual pre-training, which implicitly aligns the hidden representations of various languages, and translate-test, which explicitly translates different languages into an intermediate language, such as English. Translate-test offers better interpretability compared to multilingual pre-training. However, it has lower performance than multilingual pre-training(Conneau and Lample, 2019; Conneau et al, 2020) and struggles with word-level tasks due to translation altering word order. As a result, we propose a new Machine-created Universal Language (MUL) as an alternative intermediate language. MUL comprises a set of discrete symbols forming a universal vocabulary and a natural language to MUL translator for converting multiple natural languages to MUL. MUL unifies shared concepts from various languages into a single universal word, enhancing cross-language transfer. Additionally, MUL retains language-specific words and word order, allowing the model to be easily applied to word-level tasks. Our experiments demonstrate that translating into MUL yields improved performance compared to multilingual pre-training, and our analysis indicates that MUL possesses strong interpretability. The code is at: https://github.com/microsoft/Unicoder/tree/master/MCUL.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
