Pre-trained Programming Language Models (PPLMs) achieved many recent states of the art results for many code-related software engineering tasks. Though some studies use data flow or propose tree-based models that utilize Abstract Syntax Tree (AST), most PPLMs do not fully utilize the rich syntactical information in source code. Still, the input is considered a sequence of tokens. There are two issues; the first is computational inefficiency due to the quadratic relationship between input length and attention complexity. Second, any syntactical information, when needed as an extra input to the current PPLMs, requires the model to be pre-trained from scratch, wasting all the computational resources already used for pre-training the current models. In this work, we propose Named Entity Recognition (NER) adapters, lightweight modules that can be inserted into Transformer blocks to learn type information extracted from the AST. These adapters can be used with current PPLMs such as CodeBERT, GraphCodeBERT, and CodeT5. We train the NER adapters using a novel Token Type Classification objective function (TTC). We insert our proposed work in CodeBERT, building CodeBERTER, and evaluate the performance on two tasks of code refinement and code summarization. CodeBERTER improves the accuracy of code refinement from 16.4 to 17.8 while using 20% of training parameter budget compared to the fully fine-tuning approach, and the BLEU score of code summarization from 14.75 to 15.90 while reducing 77% of training parameters compared to the fully fine-tuning approach.
AbstractDifferentiation.jl: Backend-Agnostic Differentiable Programming in Julia
University of Basel(巴塞尔大学);Pumas-AI Inc.(普马斯人工智能公司);UNSW Canberra(新南威尔士大学堪培拉分校);Invenia Labs(因维尼亚实验室);Massachusetts Institute of Technology(麻省理工学院);Julia Computing Inc.(朱莉娅计算公司)
成功的跨语言克隆检测可以使研究人员和开发人员创建强大的语言迁移工具,一旦掌握了一种程序设计语言,就可以方便地学习其他语言,并促进代码片段在更广泛的代码库中的重用。然而,识别跨语言克隆提出了特殊的挑战,克隆检测问题。任意语言之间缺乏公共的底层表示意味着检测克隆需要以下解决方案之一:1)跨每个目标语言复制的静态分析框架,其注释匹配所有语言的语言特征,或者2)基于运行时行为检测克隆的动态分析框架。在这项工作中,我们证明了后者的解决方案,一个动态分析方法称为SLACC跨语言克隆检测的可行性。与之前的克隆检测技术一样,我们使用输入/输出行为来匹配克隆,尽管我们通过放大输入的数量和覆盖更多的数据类型来克服之前工作的局限性;因此,实现了比之前尝试更好的聚类。由于聚类是基于输入/输出行为生成的,因此SLACC支持跨语言克隆检测。作为一个额外的挑战,我们的目标是静态类型语言Java和动态类型语言Python。与最近的Java克隆检测工具HitoshiIO相比,SLACC检索的集群数量是HitoshiIO的6倍,并且精度更高(86.7% vs. 30.7%)。这是第一个对动态类型语言执行克隆检测的工作(精度= 87.3%),也是第一个对缺乏公共底层表示的语言执行克隆检测的工作(精度= 94.1%)。它为实现可伸缩语言迁移工具的更大目标迈出了第一步。
MatchFixAgent: Language-Agnostic Autonomous Repository-Level Code Translation Validation and Repair
Siebel School of Computing and Data Science, University of Illinois Urbana-Champaign, Urbana, IL, USA(伊利诺伊大学厄巴纳-香槟分校Siebel计算与数据科学学院);Amazon, Arlington, VA, USA(亚马逊公司,阿灵顿,弗吉尼亚州,美国);University of Oxford, Oxford, UK(牛津大学,牛津,英国)
Denoising Programming Knowledge Tracing with a Code Graph-based Tuning Adaptor
State Key Laboratory of Cognitive Intelligence, University of Science and Technology of China(认知智能国家重点实验室,中国科学技术大学);Institute of Artificial Intelligence, Hefei Comprehensive National Science Center(人工智能研究院,合肥综合性国家科学中心)
Doc2Spec: Synthesizing Formal Programming Specifications from Natural Language via Grammar Induction
The Pennsylvania State University(宾夕法尼亚州立大学);University of the Chinese Academy of Sciences(中国科学院大学);Institute of Computing Technology, Chinese Academy of Sciences(中国科学院计算技术研究所)