arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2010.07987cs.LGcs.CLcs.SE
收藏

Source Code用Transformers的实证研究

Empirical Study of Transformers for Source Code

HSE University(高等经济大学)

Nadezhda Chirkova, Sergey Troshin

摘要:

最初是为自然语言处理(NLP)而开发的,由于源代码和文本之间的格式相似性,Transformers现在广泛用于源代码处理。与自然语言相比,源代码是严格结构化的,即,它遵循程序设计语言的语法。最近的几项工作开发了Transformer修改,用于捕获源代码中的语法信息。这些作品的缺点是它们不相互比较,考虑不同的任务。在这项工作中,我们进行了彻底的实证研究的能力,Transformers利用句法信息在不同的任务。我们考虑三个任务(代码完成,函数命名和错误修复),并在一个统一的框架中重新实现不同的语法捕获修改。我们表明,Transformers能够作出有意义的预测纯粹基于语法信息,并强调考虑到语法信息的最佳做法,以提高模型的性能。

英文摘要:

Initially developed for natural language processing (NLP), Transformers are now widely used for source code processing, due to the format similarity between source code and text. In contrast to natural language, source code is strictly structured, i.e., it follows the syntax of the programming language. Several recent works develop Transformer modifications for capturing syntactic information in source code. The drawback of these works is that they do not compare to each other and consider different tasks. In this work, we conduct a thorough empirical study of the capabilities of Transformers to utilize syntactic information in different tasks. We consider three tasks (code completion, function naming and bug fixing) and re-implement different syntax-capturing modifications in a unified framework. We show that Transformers are able to make meaningful predictions based purely on syntactic information and underline the best practices of taking the syntactic information into account for improving the performance of the model.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑