对比提示通过推理时调控增强大语言模型中的句子嵌入
Contrastive Prompting Enhances Sentence Embeddings in LLMs through Inference-Time Steering
AI总结：
针对大语言模型句子嵌入最后词元含过多非必要信息的问题，提出即插即用的对比提示推理调控方法，通过辅助提示对比引导编码核心语义，在多类任务和不同LLM上均提升了现有提示方法的性能。
摘要：
从大语言模型（LLMs）中提取句子嵌入是一个实用方向，因为它既不需要额外数据也不需要微调。以往研究通常聚焦于提示工程，引导LLMs将句子的核心语义信息编码到最后一个词元的嵌入中。然而，这些方法中的最后一个词元仍会编码过多非必要信息（如停用词），限制了其编码能力。为此，我们提出对比提示（Contrastive Prompting, CP）方法，引入额外的辅助提示以生成更优的句子嵌入。通过与辅助提示对比，CP能够调控现有提示，使其编码句子的核心语义而非非必要信息。CP是一种即插即用的推理时干预方法，可与多种基于提示的方法结合。在语义文本相似度（STS）任务和下游分类任务上的大量实验表明，我们的方法能在不同LLMs上提升现有基于提示方法的性能。我们的代码将发布于https://github.com/zifengcheng/CP。
英文摘要：
Extracting sentence embeddings from large language models (LLMs) is a practical direction, as it requires neither additional data nor fine-tuning. Previous studies usually focus on prompt engineering to guide LLMs to encode the core semantic information of the sentence into the embedding of the last token. However, the last token in these methods still encodes an excess of non-essential information, such as stop words, limiting its encoding capacity. To this end, we propose a Contrastive Prompting (CP) method that introduces an extra auxiliary prompt to elicit better sentence embedding. By contrasting with the auxiliary prompt, CP can steer existing prompts to encode the core semantics of the sentence, rather than non-essential information. CP is a plug-and-play inference-time intervention method that can be combined with various prompt-based methods. Extensive experiments on Semantic Textual Similarity (STS) tasks and downstream classification tasks demonstrate that our method can improve the performance of existing prompt-based methods across different LLMs. Our code will be released at https://github.com/zifengcheng/CP.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
