COSEA:分层注意力卷积码搜索
COSEA: Convolutional Code Search with Layer-wise Attention
摘要:
语义代码搜索旨在检索与给定自然语言查询相关的代码片段,其吸引了许多研究努力,目的是加速软件开发。大量的在线公开代码库促使人们使用深度学习技术来构建最先进的代码搜索模型。特别地,它们利用深度神经网络将代码和查询嵌入到统一的语义向量空间中,然后使用代码和查询向量之间的相似性来近似代码和查询之间的语义相关性。然而,大多数现有的研究忽略了代码的内在结构逻辑,其中确实包含了丰富的语义信息,并未能捕捉代码的内在特征。在本文中,我们提出了一种新的深度学习架构COSEA,它利用具有逐层注意力的卷积神经网络来捕获有价值代码的内在结构逻辑。为了进一步提高COSEA的学习效率,我们提出了一种对比损失的变体来训练代码搜索模型,其中地面真值代码应该与最相似的负样本区分开来。我们已经实现了COSEA的原型。对现有的Python和SQL公共数据集进行的大量实验表明,COSEA可以在代码搜索任务上比最先进的方法实现显着改进。
英文摘要:
Semantic code search, which aims to retrieve code snippets relevant to a given natural language query, has attracted many research efforts with the purpose of accelerating software development. The huge amount of online publicly available code repositories has prompted the employment of deep learning techniques to build state-of-the-art code search models. Particularly, they leverage deep neural networks to embed codes and queries into a unified semantic vector space and then use the similarity between code's and query's vectors to approximate the semantic correlation between code and the query. However, most existing studies overlook the code's intrinsic structural logic, which indeed contains a wealth of semantic information, and fails to capture intrinsic features of codes. In this paper, we propose a new deep learning architecture, COSEA, which leverages convolutional neural networks with layer-wise attention to capture the valuable code's intrinsic structural logic. To further increase the learning efficiency of COSEA, we propose a variant of contrastive loss for training the code search model, where the ground-truth code should be distinguished from the most similar negative sample. We have implemented a prototype of COSEA. Extensive experiments over existing public datasets of Python and SQL have demonstrated that COSEA can achieve significant improvements over state-of-the-art methods on code search tasks.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。