从零开始的句子嵌入对比学习
Contrastive Learning of Sentence Embeddings from Scratch
AI总结：
提出SynCSE框架，利用大语言模型合成数据（含部分标注与从零生成）进行对比学习训练句子嵌入，在相似性和重排序任务中大幅超越无监督基线，部分设置下媲美监督模型。
摘要：
对比学习一直是训练最先进句子嵌入的主导方法。以往研究通常通过使用人工标注的自然语言推理（NLI）数据，或以无监督方式利用大规模无标注句子来学习句子嵌入。然而，即使是无标注数据，由于各种原因，其在某些领域的获取也面临挑战。为解决这些问题，我们提出了SynCSE，一种利用合成数据训练句子嵌入的对比学习框架。具体而言，我们探索利用大型语言模型为对比学习合成所需的数据样本，包括：(1) 给定无标注句子生成正负标注（SynCSE-partial），以及 (2) 从零开始生成句子及其对应标注（SynCSE-scratch）。在句子相似性和重排序任务上的实验结果表明，SynCSE-partial和SynCSE-scratch均大幅优于无监督基线，且SynCSE-partial在多数设置下甚至取得了与监督模型相当的性能。
英文摘要：
Contrastive learning has been the dominant approach to train state-of-the-art sentence embeddings. Previous studies have typically learned sentence embeddings either through the use of human-annotated natural language inference (NLI) data or via large-scale unlabeled sentences in an unsupervised manner. However, even in the case of unlabeled data, their acquisition presents challenges in certain domains due to various reasons. To address these issues, we present SynCSE, a contrastive learning framework that trains sentence embeddings with synthesized data. Specifically, we explore utilizing large language models to synthesize the required data samples for contrastive learning, including (1) producing positive and negative annotations given unlabeled sentences (SynCSE-partial), and (2) generating sentences along with their corresponding annotations from scratch (SynCSE-scratch). Experimental results on sentence similarity and reranking tasks indicate that both SynCSE-partial and SynCSE-scratch greatly outperform unsupervised baselines, and SynCSE-partial even achieves comparable performance to the supervised models in most settings.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
