arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2409.14644cs.SEcs.AI
收藏

LSem2Vec:一种简单且有效的源代码嵌入两阶段方法

LSem2Vec: A Simple yet Effective Two-Stage Approach for Source Code Embedding

Macau University of Science and Technology(澳门科学技术大学);Macau University of Science and Technology Zhuhai MUST Science and Technology Research Institute(澳门科学技术大学珠海 MUST 科技研究院);Nanjing University(南京大学)

Zixiang Xian, Chenhui Cui, Rubing Huang, Chunrong Fang, Zhenyu Chen

AI总结:

LSem2Vec是结合LLM与句子嵌入模型的两阶段源代码嵌入方法,无需特定任务训练,在三个多语言数据集上优于五种无监督基线,可推进软件工程领域发展。

摘要:

大型语言模型(LLMs)的出现极大地推动了软件工程领域的人工智能发展,源代码嵌入在代码克隆检测、代码聚类等任务中发挥着关键作用。然而,现有的源代码嵌入方法(包括基于LLMs的方法)往往依赖于代价高昂的监督训练或领域适配的微调。本文提出了LSem2Vec(LLM提取的代码语义到向量嵌入),一种结合大型语言模型与句子嵌入模型的简单且有效的源代码嵌入两阶段方法。具体而言,LSem2Vec利用LLM提取源代码的语义,再通过句子嵌入模型生成表示向量。与现有方法相比,LSem2Vec无需特定任务的训练或微调,且能有效处理LLM生成输出中常见的错误信息。为评估LSem2Vec的性能,我们在涵盖不同编程语言的三个数据集上,使用多种LLMs和句子嵌入模型开展了一系列实验。实验结果表明,LSem2Vec在五种最先进的无监督方法中表现出有效性和优越性。我们的研究结果凸显了LSem2Vec通过为源代码嵌入任务提供鲁棒且高效的解决方案,推动软件工程领域发展的潜力。

英文摘要:

The advent of large language models (LLMs) has significantly advanced artificial intelligence in software engineering, with source code embeddings playing a crucial role in tasks such as source code clone detection and source code clustering. However, existing methods for source code embedding, including those based on LLMs, often rely on costly supervised training or fine-tuning for domain adaptation. This paper proposes LSem2Vec (LLM-extracted code Semantics to Vector embedding), a simple yet effective two-stage approach to embedding source code by combining large language and sentence embedding models. Specifically, LSem2Vec leverages an LLM to extract the semantics of source code, and then uses a sentence embedding model to generate representation vectors. Compared with previous approaches, LSem2Vec eliminates the need for task-specific training or fine-tuning and effectively addresses erroneous information commonly found in LLM-generated outputs. To evaluate the performance of LSem2Vec, we conducted a series of experiments across three datasets spanning different programming languages, using various LLMs and sentence embedding models. The experimental results demonstrate the effectiveness and superiority of LSem2Vec over five state-of-the-art unsupervised methods. Our findings highlight the potential of LSem2Vec to advance the field of software engineering by providing robust and efficient solutions for source code embedding tasks.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑