D2CSE：面向对比句嵌入的差异感知深度连续提示
D2CSE: Difference-aware Deep continuous prompts for Contrastive Sentence Embeddings
AI总结：
D2CSE 通过自引导的对比学习和条件替换词元检测优化单个 PLM 上的连续提示，以约 1% 的训练参数在七个 STS 基准上提升句嵌入质量并有效区分细微语义差异。
摘要：
本文描述了用于学习句嵌入的差异感知深度连续提示对比句嵌入方法（D2CSE）。与最先进的方法相比，D2CSE 通过采用一种简单的连续提示神经架构，计算出的句向量在区分相似句子中的细微差异方面表现出色。与现有需要多个预训练语言模型（PLM）来处理原始句子和损坏（细微修改）句子对的架构不同，D2CSE 避免了多个 PLM 的繁琐微调，仅通过执行多项任务来优化连续提示——即对比学习和条件替换词元检测，全部以自引导方式进行。D2CSE 将单个 PLM 重载于连续提示上，从而大大节省了内存消耗。D2CSE 中的训练参数数量减少到现有方法的大约 1%，同时显著提高了句嵌入的质量。我们在七个语义文本相似性（STS）基准上评估 D2CSE，使用三种不同的指标，即 Spearman 秩相关系数、检索任务中的 recall@K，以及以对齐性和均匀性衡量的嵌入空间各向异性。我们的实证结果表明，浅层（并非过于精心设计的）连续提示可以有效地针对多个 NLP 任务进行打磨，并在现有最先进方法的基础上带来改进。
英文摘要：
This paper describes Difference-aware Deep continuous prompt for Contrastive Sentence Embeddings (D2CSE) that learns sentence embeddings. Compared to state-of-the-art approaches, D2CSE computes sentence vectors that are exceptional to distinguish a subtle difference in similar sentences by employing a simple neural architecture for continuous prompts. Unlike existing architectures that require multiple pretrained language models (PLMs) to process a pair of the original and corrupted (subtly modified) sentences, D2CSE avoids cumbersome fine-tuning of multiple PLMs by only optimizing continuous prompts by performing multiple tasks -- i.e., contrastive learning and conditional replaced token detection all done in a self-guided manner. D2CSE overloads a single PLM on continuous prompts and greatly saves memory consumption as a result. The number of training parameters in D2CSE is reduced to about 1\% of existing approaches while substantially improving the quality of sentence embeddings. We evaluate D2CSE on seven Semantic Textual Similarity (STS) benchmarks, using three different metrics, namely, Spearman's rank correlation, recall@K for a retrieval task, and the anisotropy of an embedding space measured in alignment and uniformity. Our empirical results suggest that shallow (not too meticulously devised) continuous prompts can be honed effectively for multiple NLP tasks and lead to improvements upon existing state-of-the-art approaches.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
