arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2503.08161cs.CLcs.IR
收藏

OASIS:面向改进代码搜索的次序增强策略

OASIS: Order-Augmented Strategy for Improved Code Search

The Hong Kong Polytechnic University(香港理工大学);Southern University of Science and Technology(南方科技大学);Kuaishou Technology(快手科技)

Zuchen Gao, Zizheng Zhan, Xianming Li, Erxin Yu, Ziqi Zhan, Haotian Zhang, Bin Chen, Yuqun Zhang, Jing Li

AI总结:

针对代码搜索中仅比较正负样本主要差异难以捕获深层语义的问题,提出OASIS策略,利用次序相似性标签训练模型捕捉负样本间细微差异,显著超越现有最先进模型。

摘要:

代码嵌入捕获代码的语义表示,对于代码搜索等各种与代码相关的大语言模型(LLM)应用至关重要。以往的训练主要依赖通过将正自然语言(NL)-代码对与批内负样本进行比较来优化InfoNCE损失。然而,由于代码上下文的稀疏性,仅通过比较正负样本对之间的主要差异进行训练可能无法捕获更深层的语义细微差别。为解决这一问题,我们提出了一种新颖的面向改进代码搜索的次序增强策略(OASIS)。它利用基于次序的相似性标签来训练模型,以捕获负样本对之间相似性的细微差异。广泛的基准评测表明,我们的OASIS模型显著优于以往仅关注主要正负差异的最先进模型。这凸显了利用带有次序标签的负样本对之间的细微差异进行有效代码嵌入训练的价值。

英文摘要:

Code embeddings capture the semantic representations of code and are crucial for various code-related large language model (LLM) applications, such as code search. Previous training primarily relies on optimizing the InfoNCE loss by comparing positive natural language (NL)-code pairs with in-batch negatives. However, due to the sparse nature of code contexts, training solely by comparing the major differences between positive and negative pairs may fail to capture deeper semantic nuances. To address this issue, we propose a novel order-augmented strategy for improved code search (OASIS). It leverages order-based similarity labels to train models to capture subtle differences in similarity among negative pairs. Extensive benchmark evaluations demonstrate that our OASIS model significantly outperforms previous state-of-the-art models focusing solely on major positive-negative differences. It underscores the value of exploiting subtle differences among negative pairs with order labels for effective code embedding training.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑