arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2108.01585cs.SE
收藏

使用Transformer模型进行代码补全的实证研究

An Empirical Study on the Usage of Transformer Models for Code Completion

Matteo Ciniselli, Nathan Cooper, Luca Pascarella, Antonio Mastropaolo, Emad Aghajani, Denys Poshyvanyk, Massimiliano Di Penta, Gabriele Bavota

备注：
arXiv admin note: substantial text overlap with arXiv:2103.07115
阅读 PDF
arXiv 页面 ↗HTML 原文 ↗

摘要：

代码完成旨在通过预测开发人员可能编写的下一个代码令牌来加速代码编写。这一领域的工作重点是提高生成的预测的准确性，深度学习（DL）模型使其实现了实质性的飞跃。然而，代码完成技术大多在预测下一个要键入的标记的场景中进行评估，很少有例外将边界推到整个代码语句的预测。因此，在更具挑战性的情况下，例如，必须生成整个代码块，对最先进的代码完成方法的性能知之甚少。我们提出了一项大规模的研究，探索最先进的基于transformer的模型在不同粒度级别支持代码完成的能力，包括单个令牌，一个或多个完整的语句，直到整个代码块（例如，for循环的迭代块）。我们试验了两个最近提出的基于transformer的模型的几个变体，即RoBERTa和文本到文本传输Transformer（T5），用于代码完成任务。所取得的结果表明，基于transformer的模型，特别是T5，代表了代码完成的可行解决方案，完美的预测范围从约29%，当要求模型猜测整个块时，达到约69%，在更简单的情况下，从相同的代码语句中屏蔽几个令牌。

英文摘要：

Code completion aims at speeding up code writing by predicting the next code token(s) the developer is likely to write. Works in this field focused on improving the accuracy of the generated predictions, with substantial leaps forward made possible by deep learning (DL) models. However, code completion techniques are mostly evaluated in the scenario of predicting the next token to type, with few exceptions pushing the boundaries to the prediction of an entire code statement. Thus, little is known about the performance of state-of-the-art code completion approaches in more challenging scenarios in which, for example, an entire code block must be generated. We present a large-scale study exploring the capabilities of state-of-the-art Transformer-based models in supporting code completion at different granularity levels, including single tokens, one or multiple entire statements, up to entire code blocks (e.g., the iterated block of a for loop). We experimented with several variants of two recently proposed Transformer-based models, namely RoBERTa and the Text-To-Text Transfer Transformer (T5), for the task of code completion. The achieved results show that Transformer-based models, and in particular the T5, represent a viable solution for code completion, with perfect predictions ranging from ~29%, obtained when asking the model to guess entire blocks, up to ~69%, reached in the simpler scenario of few tokens masked from the same code statement.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

↑