通过知识驱动的数据增强与高斯衰减对比学习增强无监督句子嵌入
Enhancing Unsupervised Sentence Embeddings via Knowledge-Driven Data Augmentation and Gaussian-Decayed Contrastive Learning
AI总结：
针对无监督句子嵌入中数据多样性不足、噪声高的问题，提出结合知识图谱的LLM数据增强流水线与高斯衰减对比学习GCSE模型，在STS任务上以更少数据和更小模型实现SOTA性能。
摘要：
近年来，利用大语言模型（LLMs）进行数据增已使无监督句子嵌入模型取得显著提升。然而，现有方法面临两大核心挑战：数据多样性有限与数据噪声较高。当前方法往往忽略实体、数量等细粒度知识，导致多样性不足；此外，无监督数据常缺乏判别信息，生成的合成样本可能引入噪声。本文提出一种基于流水线的LLM数据增强方法，并引入高斯衰减梯度辅助对比句子嵌入（Gaussian-decayed gradient-assisted Contrastive Sentence Embedding, GCSE）模型以增强无监督句子嵌入。为解决数据多样性低的问题，该流水线利用知识图谱（KGs）提取实体与数量，使LLM生成更多样的样本；为应对高数据噪声，GCSE模型采用高斯衰减函数限制错误难负样本的影响，提升模型的判别能力。实验结果表明，该方法在语义文本相似度（STS）任务上达到当前最优性能，且使用更少的数据样本与更小的LLM，证明其在多种模型下的高效性与鲁棒性。
英文摘要：
Recently, using large language models (LLMs) for data augmentation has led to considerable improvements in unsupervised sentence embedding models. However, existing methods encounter two primary challenges: limited data diversity and high data noise. Current approaches often neglect fine-grained knowledge, such as entities and quantities, leading to insufficient diversity. Besides, unsupervised data frequently lacks discriminative information, and the generated synthetic samples may introduce noise. In this paper, we propose a pipeline-based data augmentation method via LLMs and introduce the Gaussian-decayed gradient-assisted Contrastive Sentence Embedding (GCSE) model to enhance unsupervised sentence embeddings. To tackle the issue of low data diversity, our pipeline utilizes knowledge graphs (KGs) to extract entities and quantities, enabling LLMs to generate more diverse samples. To address high data noise, the GCSE model uses a Gaussian-decayed function to limit the impact of false hard negative samples, enhancing the model's discriminative capability. Experimental results show that our approach achieves state-of-the-art performance in semantic textual similarity (STS) tasks, using fewer data samples and smaller LLMs, demonstrating its efficiency and robustness across various models.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
