利用最优传输作为对齐目标来微调多语言上下文嵌入
Using Optimal Transport as Alignment Objective for fine-tuning Multilingual Contextualized Embeddings
AI总结：
本文提出在微调过程中使用最优传输作为对齐目标，以提升多语言上下文表示，无需预训练词对齐，通过上下文无监督学习词对齐，实现不同映射类型，实验在XNLI和XQuAD任务上取得优于基线的改进。
摘要：
最近的研究提出了不同的方法来改进上下文化设置下的多语言词表示，包括在源和目标嵌入空间之间对齐的技术。对于上下文化嵌入，对齐变得更加复杂，因为还需要考虑上下文。在本文中，我们提出在微调过程中使用最优传输（OT）作为对齐目标，以进一步改进多语言上下文表示，以适应下游跨语言转移任务。这种方法不需要在微调前的词对齐对，这可能会导致次优匹配，而是通过上下文无监督地学习词对齐。它还允许不同的映射类型，因为源和目标句子之间有软匹配。我们在此两个任务（XNLI和XQuAD）上进行了基准测试，并在基线之上实现了改进，同时与最近的类似工作相比取得了具有竞争力的结果。
英文摘要：
Recent studies have proposed different methods to improve multilingual word representations in contextualized settings including techniques that align between source and target embedding spaces. For contextualized embeddings, alignment becomes more complex as we additionally take context into consideration. In this work, we propose using Optimal Transport (OT) as an alignment objective during fine-tuning to further improve multilingual contextualized representations for downstream cross-lingual transfer. This approach does not require word-alignment pairs prior to fine-tuning that may lead to sub-optimal matching and instead learns the word alignments within context in an unsupervised manner. It also allows different types of mappings due to soft matching between source and target sentences. We benchmark our proposed method on two tasks (XNLI and XQuAD) and achieve improvements over baselines as well as competitive results compared to similar recent works.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
