HiCL：无监督句子嵌入的分层对比学习
HiCL: Hierarchical Contrastive Learning of Unsupervised Sentence Embeddings
AI总结：
HiCL通过分层对比学习同时建模局部片段与全局序列关系，提升无监督句子嵌入的训练效率与效果，在七个STS任务上超越SNCSE模型。
摘要：
在本文中，我们提出了一种分层对比学习框架HiCL，该框架同时考虑局部片段级和全局序列级关系，以提高训练效率和有效性。传统方法通常将整个序列编码后与其他序列进行对比，往往忽略了局部表示学习，导致在泛化到较短文本时面临挑战。相反，HiCL通过将序列划分为多个片段，并采用局部和全局对比学习来建模片段级和序列级关系，从而提升了其有效性。此外，考虑到Transformer在输入令牌上的二次时间复杂度，HiCL通过首先编码短片段，然后聚合它们以获得序列表示，从而提高了训练效率。大量实验表明，HiCL在七个广泛评估的STS任务上提升了先前最优的SNCSE模型，在BERT-large上平均提升+0.2%，在RoBERTa-large上平均提升+0.44%。
英文摘要：
In this paper, we propose a hierarchical contrastive learning framework, HiCL, which considers local segment-level and global sequence-level relationships to improve training efficiency and effectiveness. Traditional methods typically encode a sequence in its entirety for contrast with others, often neglecting local representation learning, leading to challenges in generalizing to shorter texts. Conversely, HiCL improves its effectiveness by dividing the sequence into several segments and employing both local and global contrastive learning to model segment-level and sequence-level relationships. Further, considering the quadratic time complexity of transformers over input tokens, HiCL boosts training efficiency by first encoding short segments and then aggregating them to obtain the sequence representation. Extensive experiments show that HiCL enhances the prior top-performing SNCSE model across seven extensively evaluated STS tasks, with an average increase of +0.2% observed on BERT-large and +0.44% on RoBERTa-large.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
