在对比学习中利用多语言正例以改进句子嵌入
Leveraging Multi-lingual Positive Instances in Contrastive Learning to Improve Sentence Embedding
AI总结：
针对多语言句子嵌入学习，本研究提出MPCL方法，通过利用多个多语言正例的跨语言优势与传递相似性改进对比学习，在多类任务及未见过语言的跨语言迁移上均优于传统单正例对比学习。
摘要：
学习多语言句子嵌入是自然语言处理中的一项基础任务。当前单语言与多语言句子嵌入的学习趋势主要基于锚点、一个正例和多个负例之间的对比学习（CL）。在本研究中，我们认为多语言句子嵌入应当考虑利用多个正例，原因有二：（1）多语言的正例有助于跨语言学习；（2）多个正例之间的传递相似性能为学习提供可靠的结构信息。为探究对比学习中多个正例的作用，我们提出了一种名为MPCL的新方法，可有效利用多个正例改进多语言句子嵌入的学习。在多种骨干模型和下游任务上的实验结果表明，与传统对比学习相比，MPCL在检索、语义相似度和分类任务上均取得了更优性能。我们还发现，在未见过的语言中，使用多个正例训练的句子嵌入模型比使用单个正例训练的模型表现出更好的跨语言迁移性能。
英文摘要：
Learning multi-lingual sentence embeddings is a fundamental task in natural language processing. Recent trends in learning both mono-lingual and multi-lingual sentence embeddings are mainly based on contrastive learning (CL) among an anchor, one positive, and multiple negative instances. In this work, we argue that leveraging multiple positives should be considered for multi-lingual sentence embeddings because (1) positives in a diverse set of languages can benefit cross-lingual learning, and (2) transitive similarity across multiple positives can provide reliable structural information for learning. In order to investigate the impact of multiple positives in CL, we propose a novel approach, named MPCL, to effectively utilize multiple positive instances to improve the learning of multi-lingual sentence embeddings. Experimental results on various backbone models and downstream tasks demonstrate that MPCL leads to better retrieval, semantic similarity, and classification performances compared to conventional CL. We also observe that in unseen languages, sentence embedding models trained on multiple positives show better cross-lingual transfer performance than models trained on a single positive instance.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
