基于AI反馈改进句子嵌入的对比学习
Improving Contrastive Learning of Sentence Embeddings from AI Feedback
AI总结：
本文提出CLAIF方法，利用大型预训练语言模型的AI反馈构建带细粒度相似度分数的样本对，并结合人类反馈提供监督信号，显著提升了句子嵌入对比学习在语义文本相似性等任务上的性能。
摘要：
对比学习已成为自然语言处理中的流行方法，尤其适用于句子嵌入的学习。然而，自然语言的离散性使得难以确保通过数据增强方法生成的正负样本对的质量。尽管监督对比学习能通过人类反馈标签生成更准确的样本对，但仍缺乏细粒度的训练信号。本文中，我们提出基于AI反馈的对比学习（CLAIF），以改进句子嵌入的对比学习。该方法利用来自大型预训练语言模型（LLMs）的AI反馈，构建带有细粒度样本相似度分数的样本对，从而改进对比学习。此外，我们结合人类反馈与AI反馈，为句子嵌入的监督对比学习提供更好的监督信号。实验结果表明，与其他无监督和监督对比学习方法相比，我们的方法在多个语义文本相似性（STS）和迁移学习任务上取得了最先进的性能。
英文摘要：
Contrastive learning has become a popular approach in natural language processing, particularly for the learning of sentence embeddings. However, the discrete nature of natural language makes it difficult to ensure the quality of positive and negative sample pairs generated through data augmentation methods. Although supervised contrastive learning can produce more accurate sample pairs with human feedback labels, it still lacks fine-grained training signals. In this paper, we propose to improve \textbf{C}ontrastive \textbf{L}earning of sentence embeddings from \textbf{AI} \textbf{F}eedback \textbf{(CLAIF)}. Our method utilizes AI feedback from large pre-trained language models (LLMs) to construct sample pairs with fine-grained sample similarity scores to improve contrastive learning. Besides, we combine human feedback and AI feedback to provide better supervision signals for supervised contrastive learning of sentence embeddings. Experimental results show that our method achieves state-of-the-art performance on several semantic textual similarity (STS) and transfer learning tasks compared to other unsupervised and supervised contrastive learning methods.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
