Learning distributed representations of source code has been a challenging task for machine learning models. Earlier works treated programs as text so that natural language methods can be readily applied. Unfortunately, such approaches do not capitalize on the rich structural information possessed by source code. Of late, Graph Neural Network (GNN) was proposed to learn embeddings of programs from their graph representations. Due to the homogeneous and expensive message-passing procedure, GNN can suffer from precision issues, especially when dealing with programs rendered into large graphs. In this paper, we present a new graph neural architecture, called Graph Interval Neural Network (GINN), to tackle the weaknesses of the existing GNN. Unlike the standard GNN, GINN generalizes from a curated graph representation obtained through an abstraction method designed to aid models to learn. In particular, GINN focuses exclusively on intervals for mining the feature representation of a program, furthermore, GINN operates on a hierarchy of intervals for scaling the learning to large graphs. We evaluate GINN for two popular downstream applications: variable misuse prediction and method name prediction. Results show in both cases GINN outperforms the state-of-the-art models by a comfortable margin. We have also created a neural bug detector based on GINN to catch null pointer deference bugs in Java code. While learning from the same 9,000 methods extracted from 64 projects, GINN-based bug detector significantly outperforms GNN-based bug detector on 13 unseen test projects. Next, we deploy our trained GINN-based bug detector and Facebook Infer to scan the codebase of 20 highly starred projects on GitHub. Through our manual inspection, we confirm 38 bugs out of 102 warnings raised by GINN-based bug detector compared to 34 bugs out of 129 warnings for Facebook Infer.
SCELMo: Source Code Embeddings from Language Models
The University of Edinburgh(爱丁堡大学);Google Research(谷歌研究院)
在计算机程序中连续嵌入令牌已被用于支持各种软件开发工具,包括可读性、代码搜索和程序修复。上下文嵌入在自然语言处理中很常见,但以前没有在软件工程中应用。我们引入了一套新的深度上下文化的词表示的计算机程序的基础上语言模型。我们使用Peters et al(2018)的ELMo(embeddings from语言模型)框架训练了一组嵌入。我们调查这些嵌入是否是有效的,当微调的下游任务的错误检测。我们表明,即使是在相对较小的程序语料库上训练的低维嵌入也可以提高最先进的机器学习系统的缺陷检测能力。
Learning Program Embeddings to Propagate Feedback on Student Code
Stanford University(斯坦福大学)
提供反馈,既评估最终的工作,又给卡住的学生提供提示,对于从数千到数百万学生的大规模在线课程中的开放式作业来说是很困难的。我们引入了一个神经网络的方法来编码程序作为一个线性映射从嵌入的前置空间到嵌入的后置空间,并提出了一个算法的反馈规模使用这些线性映射作为功能。我们将我们的算法应用于Code.org Hour of Code和斯坦福大学CS1课程的评估,在那里我们将人类对学生作业的评论传播到数量级更多的提交。
Embedding API Dependency Graph for Neural Code Generation
Shandong Normal University(山东师范大学);The University of Newcastle(纽卡斯尔大学);Big Data Center of Shandong Province(山东省大数据中心);Institute of Information Engineering, Chinese Academy of Sciences(中国科学院信息工程研究所);University of Chinese Academy of Sciences(中国科学院大学)