arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2601.05075cs.CL
收藏

SemPA:通过语义偏好对齐提升大语言模型的句子嵌入

SemPA: Improving Sentence Embeddings of Large Language Models through Semantic Preference Alignment

College of Computer Science and Software Engineering, Shenzhen University(深圳大学计算机科学与软件工程学院);School of Computer Science and Electronic Engineering, University of Surrey(Surrey大学计算机科学与电子工程学院)

Ziyang Chen, Zhenxuan Huang, Yile Wang, Weiqin Wang, Lu Yin, Hui Huang

AI总结:

SemPA通过语义偏好对齐提升大语言模型的句子嵌入能力,同时保持其生成能力。

摘要:

传统句子嵌入方法在非生成性预训练模型上采用词级对比学习。最近,基于生成性大语言模型(LLM)的嵌入方法逐渐出现。这些方法要么依赖于固定的提示模板,要么涉及对模型架构的修改。前者缺乏对模型的进一步优化,导致性能有限,而后者改变模型内部的计算机制,从而影响其生成能力。我们提出了SemPA,一种通过语义偏好对齐来提升句子表示同时保持LLM生成能力的新方法。我们利用句子级的直接偏好优化(DPO)高效优化在改写生成任务上的LLM,其中模型学习区分语义等价的句子同时保持其固有的生成能力。理论上,我们建立了DPO与在Plackett-Luce模型框架下对比学习之间的正式联系。实验上,在语义文本相似性任务和各种LLM基准测试中,结果表明SemPA在不牺牲LLM固有生成能力的情况下实现了更好的语义表示。

英文摘要:

Traditional sentence embedding methods employ token-level contrastive learning on non-generative pre-trained models. Recently, there have emerged embedding methods based on generative large language models (LLMs). These methods either rely on fixed prompt templates or involve modifications to the model architecture. The former lacks further optimization of the model and results in limited performance, while the latter alters the internal computational mechanisms of the model, thereby compromising its generative capabilities. We propose SemPA, a novel approach that boosts the sentence representations while preserving the generative ability of LLMs via semantic preference alignment. We leverage sentence-level Direct Preference Optimization (DPO) to efficiently optimize LLMs on a paraphrase generation task, where the model learns to discriminate semantically equivalent sentences while preserving inherent generative capacity. Theoretically, we establish a formal connection between DPO and contrastive learning under the Plackett-Luce model framework. Empirically, experimental results on both semantic textual similarity tasks and various benchmarks for LLMs show that SemPA achieves better semantic representations without sacrificing the inherent generation capability of LLMs.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑