大语言模型可通过对比式优化其生成结果以实现更好的句子表示学习
Large Language Models can Contrastively Refine their Generation for Better Sentence Representation Learning
AI总结：
针对对比学习句子表示受LLM生成句子对质量限制的问题，提出三阶段优化的MultiCSR框架，可让弱模型超ChatGPT、强模型刷新SOTA。
摘要：
近年来，大语言模型（LLMs）作为一项突破性技术兴起，其无与伦比的文本生成能力引发了人们将其应用于基础句子表示学习任务的兴趣。现有方法已探索将LLMs用作数据标注器，生成合成数据以训练基于对比学习的句子嵌入模型（如SimCSE）。然而，由于对比学习模型对句子对质量较为敏感，这些方法的有效性在很大程度上受LLMs生成内容的影响，凸显了在句子表示学习场景下实现更精细化生成的必要性。基于这一前提，我们提出MultiCSR——一种多层次对比句子表示学习框架，该框架将提示LLMs生成语料以训练基础句子嵌入模型的过程分解为三个阶段（即句子生成、句子对构建、批内训练），并在这三个不同阶段对生成内容进行优化，确保仅使用高质量句子对训练基础对比学习模型。我们的大量实验表明，MultiCSR可让性能较弱的LLM超越ChatGPT的表现，而将其应用于ChatGPT时则能取得更优的当前最优（SOTA）结果。综合分析进一步凸显了我们的框架在各类应用场景中的潜力，以及借助LLMs实现更优句子表示学习的价值。
英文摘要：
Recently, large language models (LLMs) have emerged as a groundbreaking technology and their unparalleled text generation capabilities have sparked interest in their application to the fundamental sentence representation learning task. Existing methods have explored utilizing LLMs as data annotators to generate synthesized data for training contrastive learning based sentence embedding models such as SimCSE. However, since contrastive learning models are sensitive to the quality of sentence pairs, the effectiveness of these methods is largely influenced by the content generated from LLMs, highlighting the need for more refined generation in the context of sentence representation learning. Building upon this premise, we propose MultiCSR, a multi-level contrastive sentence representation learning framework that decomposes the process of prompting LLMs to generate a corpus for training base sentence embedding models into three stages (i.e., sentence generation, sentence pair construction, in-batch training) and refines the generated content at these three distinct stages, ensuring only high-quality sentence pairs are utilized to train a base contrastive learning model. Our extensive experiments reveal that MultiCSR enables a less advanced LLM to surpass the performance of ChatGPT, while applying it to ChatGPT achieves better state-of-the-art results. Comprehensive analyses further underscore the potential of our framework in various application scenarios and achieving better sentence representation learning with LLMs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
