从真实世界的数据集学习自动完成
Learning Autocompletion from Real-World Datasets
摘要：
代码完成是一种流行的软件开发工具，集成到所有主要的IDE中。许多神经语言模型在合成基准上的完成建议预测中取得了令人满意的结果。然而，最近的一项研究When Code Completion Fails：a Case Study on Real-World Completions表明，这些结果可能不会转化为现实世界性能的改善。为了对抗这种影响，我们在真实世界的代码完成示例上训练模型，发现这些模型的准确率分别比在提交的源代码和工作版本快照上训练的模型高出12.8%和13.8%。我们在建模技术中观察到这种改进，并通过A/B测试表明，它对应于程序员实际自动完成使用量增加6.2%。此外，我们的研究还描述了一个大型的自动补全使用记录语料库，以研究为什么在真实世界的例子上训练会导致更强大的模型。
英文摘要：
Code completion is a popular software development tool integrated into all major IDEs. Many neural language models have achieved promising results in completion suggestion prediction on synthetic benchmarks. However, a recent study When Code Completion Fails: a Case Study on Real-World Completions demonstrates that these results may not translate to improvements in real-world performance. To combat this effect, we train models on real-world code completion examples and find that these models outperform models trained on committed source code and working version snapshots by 12.8% and 13.8% accuracy respectively. We observe this improvement across modeling technologies and show through A/B testing that it corresponds to a 6.2% increase in programmers' actual autocompletion usage. Furthermore, our study characterizes a large corpus of logged autocompletion usages to investigate why training on real-world examples leads to stronger models.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
