arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2210.17040cs.SEcs.AI
收藏

CodeEditor:使用预训练模型学习编辑源代码

CodeEditor: Learning to Edit Source Code with Pre-trained Models

Peking University(北京大学);Zhejiang University(浙江大学)

Jia Li, Ge Li, Zhuo Li, Zhi Jin, Xing Hu, Kechi Zhang, Zhiyi Fu

AI总结:

本文提出了一种专门针对代码编辑的新型预训练任务及模型CodeEditor,通过学习将突变代码版本还原为真实代码来掌握编辑模式,在微调、少样本和零样本三种设置下均显著优于现有SOTA基线。

摘要:

在软件开发过程中,开发者经常出于各种原因(例如代码重构)执行重复的代码编辑活动。预训练的代码编辑模型已经取得了最先进(SOTA)的结果。预训练模型首先通过预训练任务进行预训练,然后通过代码编辑任务进行微调。现有的预训练任务主要是代码填充任务(例如掩码语言建模),这些任务源自自然语言处理领域,并非为自动代码编辑而设计。本文提出了一种专门针对代码编辑的新型预训练任务,并展示了一个名为CodeEditor的有效预训练代码编辑模型。我们的预训练任务进一步提升了代码编辑模型的性能和泛化能力。具体来说,我们收集了大量真实的代码片段作为真实值,并使用一个强大的生成器将其重写为突变版本。然后,我们预训练CodeEditor将突变版本编辑为相应的真实值,以学习编辑模式。我们在四个代码编辑数据集上进行了实验,并在三种设置下评估了预训练的CodeEditor。(1)在微调设置中,我们使用四个数据集训练预训练的CodeEditor,并在测试数据上对其进行评估。CodeEditor在四个数据集上分别比SOTA基线高出15%、25.5%、9.4%和26.6%。(2)在少样本设置中,我们使用有限数据训练预训练的CodeEditor,并在测试数据上对其进行评估。CodeEditor的表现显著优于所有基线。(3)在零样本设置中,CodeEditor正确编辑了1,113个程序,而SOTA基线则无法工作。

英文摘要:

Developers often perform repetitive code editing activities for various reasons (e.g., code refactoring) during software development. Pre-trained code editing models have achieved the state-of-the-art (SOTA) results. Pre-trained models are first pre-trained with pre-training tasks and fine-tuned with the code editing task. Existing pre-training tasks mainly are code infilling tasks (e.g., masked language modeling), which are derived from the natural language processing field and are not designed for automatic code editing. This paper proposes a novel pre-training task specialized in code editing and presents an effective pre-trained code editing model named CodeEditor. Our pre-training task further improves the performance and generalization ability of code editing models. Specifically, we collect lots of real-world code snippets as the ground truth and use a powerful generator to rewrite them into mutated versions. Then, we pre-train our CodeEditor to edit mutated versions into the corresponding ground truth, to learn edit patterns. We conduct experiments on four code editing datasets and evaluate the pre-trained CodeEditor in three settings. (1) In the fine-tuning setting, we train the pre-trained CodeEditor with four datasets and evaluate it on the test data. CodeEditor outperforms the SOTA baselines by 15%, 25.5%, and 9.4% and 26.6% on four datasets. (2) In the few-shot setting, we train the pre-trained CodeEditor with limited data and evaluate it on the test data. CodeEditor substantially performs better than all baselines. (3) In the zero-shot setting, CodeEditor correctly edits 1,113 programs while the SOTA baselines can not work.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑