arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2202.06689cs.SEcs.LG
收藏

CodeFill:基于结构和伪码序列联合学习的多令牌码补全算法

CodeFill: Multi-token Code Completion by Jointly Learning from Structure and Naming Sequences

Delft University of Technology(代尔夫特理工大学)

Maliheh Izadi, Roberta Gismondi, Georgios Gousios

摘要:

代码完成是IDE的一个基本功能,但当前的自动完成器仅限于基于语法或基于NLP的单个标记完成。这两种方法都有明显的缺点:基于语法的自动完成在动态类型语言环境中受到限制,而基于NLP的自动完成程序很难理解程序设计语言的语义和开发人员的代码上下文。在这项工作中,我们提出了CodeFill,一个语言模型自动完成,结合了学习的结构和命名信息。使用并行Transformer架构和多任务学习,CodeFill使用源代码标记名称序列及其等效的AST标记类型。独特的是,CodeFill针对单标记和多标记(语句)预测进行了训练,这使其能够学习语法和命名元素之间的长期依赖关系。我们在两个数据集上训练CodeFill,分别由29 M和425 M行代码组成。为了使评估更加真实,我们开发了一种方法来自动推断源代码中的完成点。我们将CodeFill与四个基线和两个最先进的模型GPT-C和TravTrans+进行了比较。CodeFill在单令牌预测方面超过了所有基线(MRR:70.9% vs. 66.2%和67.8%),并且在多令牌预测方面优于最先进的模型(ROUGE-L:63.7% vs. 52.4%和59.2%,n=4个令牌)。我们公开发布我们的源代码和数据集。

英文摘要:

Code completion is an essential feature of IDEs, yet current autocompleters are restricted to either grammar-based or NLP-based single token completions. Both approaches have significant drawbacks: grammar-based autocompletion is restricted in dynamically-typed language environments, whereas NLP-based autocompleters struggle to understand the semantics of the programming language and the developer's code context. In this work, we present CodeFill, a language model for autocompletion that combines learned structure and naming information. Using a parallel Transformer architecture and multi-task learning, CodeFill consumes sequences of source code token names and their equivalent AST token types. Uniquely, CodeFill is trained both for single-token and multi-token (statement) prediction, which enables it to learn long-range dependencies among grammatical and naming elements. We train CodeFill on two datasets, consisting of 29M and 425M lines of code, respectively. To make the evaluation more realistic, we develop a method to automatically infer points in the source code at which completion matters. We compare CodeFill against four baselines and two state-of-the-art models, GPT-C and TravTrans+.CodeFill surpasses all baselines in single token prediction (MRR: 70.9% vs. 66.2% and 67.8%) and outperforms the state of the art for multi-token prediction (ROUGE-L: 63.7% vs. 52.4% and 59.2%, for n=4 tokens). We publicly release our source code and datasets.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑