用于无监督句子嵌入的实例平滑对比学习
Instance Smoothed Contrastive Learning for Unsupervised Sentence Embedding
AI总结：
提出IS-CSE方法，通过动态记忆缓冲区检索正例嵌入组并利用自注意力聚合生成平滑实例嵌入，在标准STS任务上显著超越unsup-SimCSE基线。
摘要：
基于对比学习的方法（如unsup-SimCSE）在学习无监督句子嵌入方面取得了最先进的（SOTA）性能。然而，在以往的研究中，用于对比学习的每个嵌入仅源自一个句子实例，我们将这些嵌入称为实例级嵌入。换言之，每个嵌入被视为其自身的唯一类别，这可能会损害泛化性能。在本研究中，我们提出IS-CSE（实例平滑对比句子嵌入）以平滑特征空间中嵌入的边界。具体而言，我们根据语义相似度从动态记忆缓冲区中检索嵌入，以获取正例嵌入组。然后，通过自注意力操作聚合组中的嵌入，生成平滑的实例嵌入以供进一步分析。我们在标准语义文本相似性（STS）任务上评估了我们的方法，在BERT-base、BERT-large、RoBERTa-base和RoBERTa-large的基础上分别达到了78.30%、79.47%、77.73%和79.42%的平均Spearman相关性，相比unsup-SimCSE分别提升了2.05%、1.06%、1.16%和0.52%。
英文摘要：
Contrastive learning-based methods, such as unsup-SimCSE, have achieved state-of-the-art (SOTA) performances in learning unsupervised sentence embeddings. However, in previous studies, each embedding used for contrastive learning only derived from one sentence instance, and we call these embeddings instance-level embeddings. In other words, each embedding is regarded as a unique class of its own, whichmay hurt the generalization performance. In this study, we propose IS-CSE (instance smoothing contrastive sentence embedding) to smooth the boundaries of embeddings in the feature space. Specifically, we retrieve embeddings from a dynamic memory buffer according to the semantic similarity to get a positive embedding group. Then embeddings in the group are aggregated by a self-attention operation to produce a smoothed instance embedding for further analysis. We evaluate our method on standard semantic text similarity (STS) tasks and achieve an average of 78.30%, 79.47%, 77.73%, and 79.42% Spearman's correlation on the base of BERT-base, BERT-large, RoBERTa-base, and RoBERTa-large respectively, a 2.05%, 1.06%, 1.16% and 0.52% improvement compared to unsup-SimCSE.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
