arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2109.07173cs.SEcs.PL
收藏

代码嵌入的比较与超越

A Comparison of Code Embeddings and Beyond

Siqi Han, DongXia Wang, Wanting Li, Xuesong Lu

摘要:

程序表示学习是软件工程应用中的一项基本任务。随着“大代码”的出现和深度学习技术的发展,各种程序表示学习模型被提出来理解程序的语义属性,并应用于不同的软件工程任务。然而,以前的研究还没有全面评估这些深层模型在不同任务上的泛化能力,因此模型的利弊尚不清楚。在这篇经验论文中,我们试图通过系统地评估八个程序表示学习模型在三个常见任务上的性能来弥合这一差距,其中六个模型基于抽象语法树,两个模型基于源代码的纯文本。我们解释了选择模型和任务的标准,以及在每个任务中实现端到端学习的方法。性能评估的结果表明,它们在每个任务中执行的是分散的,并且基于AST的模型的性能在不同的任务上通常是不稳定的。为了进一步解释结果,我们应用预测归因技术来找出模型捕获的元素,并负责每个任务中的预测。基于这些发现,我们讨论了更好地捕捉源代码中信息的一些一般原则,并希望能启发研究人员改进用于软件工程任务的程序表示学习方法。

英文摘要:

Program representation learning is a fundamental task in software engineering applications. With the availability of "big code" and the development of deep learning techniques, various program representation learning models have been proposed to understand the semantic properties of programs and applied on different software engineering tasks. However, no previous study has comprehensively assessed the generalizability of these deep models on different tasks, so that the pros and cons of the models are unclear. In this experience paper, we try to bridge this gap by systemically evaluating the performance of eight program representation learning models on three common tasks, where six models are based on abstract syntax trees and two models are based on plain text of source code. We kindly explain the criteria for selecting the models and tasks, as well as the method for enabling end-to-end learning in each task. The results of performance evaluation show that they perform diversely in each task and the performance of the AST-based models is generally unstable over different tasks. In order to further explain the results, we apply a prediction attribution technique to find what elements are captured by the models and responsible for the predictions in each task. Based on the findings, we discuss some general principles for better capturing the information in the source code, and hope to inspire researchers to improve program representation learning methods for software engineering tasks.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑