通过软对比学习改进多语言对齐
Improving Multi-lingual Alignment Through Soft Contrastive Learning
AI总结：
提出软对比学习改进多语言句子对齐，利用单语言模型相似性作为软标签，在双语挖掘和STS任务上超越硬标签方法及LaBSE。
摘要：
生成高质量的多语言句子表示对于在跨语言下游任务中取得高性能至关重要。在这项工作中，我们提出了一种新颖的方法，基于预训练的单语言嵌入模型测量的句子相似性来对齐多语言嵌入。给定翻译句子对，我们训练一个多语言模型，使得跨语言嵌入之间的相似性遵循在单语言教师模型处测量的句子相似性。我们的方法可以被视为具有软标签的对比学习，其中软标签定义为句子之间的相似性。我们在五种语言上的实验结果表明，在双语文本挖掘任务和语义文本相似性（STS）任务的各种基准测试中，具有软标签的对比损失远远优于具有硬标签的传统对比损失。此外，我们的方法在Tatoeba数据集上优于现有的多语言嵌入，包括LaBSE。代码可在https://github.com/YAI12xLinq-B/IMASCL获取。
英文摘要：
Making decent multi-lingual sentence representations is critical to achieve high performances in cross-lingual downstream tasks. In this work, we propose a novel method to align multi-lingual embeddings based on the similarity of sentences measured by a pre-trained mono-lingual embedding model. Given translation sentence pairs, we train a multi-lingual model in a way that the similarity between cross-lingual embeddings follows the similarity of sentences measured at the mono-lingual teacher model. Our method can be considered as contrastive learning with soft labels defined as the similarity between sentences. Our experimental results on five languages show that our contrastive loss with soft labels far outperforms conventional contrastive loss with hard labels in various benchmarks for bitext mining tasks and STS tasks. In addition, our method outperforms existing multi-lingual embeddings including LaBSE, for Tatoeba dataset. The code is available at https://github.com/YAI12xLinq-B/IMASCL
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
