Code completion aims to help improve developers' productivity by suggesting the next code tokens from a given context. Various approaches have been proposed to incorporate abstract syntax tree (AST) information for model training, ensuring that code completion is aware of the syntax of the programming languages. However, existing syntax-aware code completion approaches are not on-the-fly, as we found that for every two-thirds of characters that developers type, AST fails to be extracted because it requires the syntactically correct source code, limiting its practicality in real-world scenarios. On the other hand, existing on-the-fly code completion does not consider syntactic information yet. In this paper, we propose PyCoder to leverage token types, a kind of lightweight syntactic information, which is readily available and aligns with the natural order of source code. Our PyCoder is trained in a multi-task training manner so that by learning the supporting task of predicting token types during the training phase, the models achieve better performance on predicting tokens and lines of code without the need for token types in the inference phase. Comprehensive experiments show that PyCoder achieves the first rank on the CodeXGLUE leaderboard with an accuracy of 77.12% for the token-level predictions, which is 0.43%-24.25% more accurate than baselines. In addition, PyCoder achieves an exact match of 43.37% for the line-level predictions, which is 3.63%-84.73% more accurate than baselines. These results lead us to conclude that token type information (an alternative to syntactic information) that is rarely used in the past can greatly improve the performance of code completion approaches, without requiring the syntactically correct source code like AST-based approaches do. Our PyCoder is publicly available on HuggingFace and GitHub.
Comparative Analysis of Large Language Models for Context-Aware Code Completion using SAFIM Framework
University of California San Diego(加利福尼亚大学圣迭戈分校);Simon Fraser University(西蒙菲莎大学);Duke University(杜克大学);University of Amsterdam(阿姆斯特丹大学);Johns Hopkins University(约翰斯·霍普金斯大学)
What Do They Capture? -- A Structural Analysis of Pre-Trained Language Models for Source Code
National Engineering Research Center for Big Data Technology and System, Services Computing Technology and System Lab, Cluster and Grid Computing Lab, HUST(华中科技大学(HUST)大数据技术与系统国家工程研究中心、服务计算技术与系统实验室、集群与网格计算实验室);School of Computer Science and Technology, Huazhong University of Science and Technology(华中科技大学计算机科学与技术学院);University of Newcastle(纽卡斯尔大学);School of Computer Science, University of Technology Sydney(悉尼科技大学计算机学院)
在本文中,我们提出了一种新的端到端的AI辅助代码完成方法,称为Pythia。它生成方法和API建议的排名列表,可供软件开发人员在编辑时使用。该系统目前部署为Visual Studio Code IDE中Intellicode扩展的一部分。Pythia利用了最先进的大规模深度学习模型,这些模型是在从抽象语法树中提取的代码上下文上训练的。它被设计为在高吞吐量下工作,预测100个$ms$数量级的最佳匹配代码完成。我们描述了该系统的架构,执行比较基于频率的方法和基于调用的马尔可夫链语言模型,并讨论了轻量级客户端设备上的Pythia模型服务的挑战。在2700个Python开源软件GitHub存储库上获得的离线评估结果显示,对于项目内和跨项目设置,前5名的准确率为92%,超过基线模型20%。
Teaching LLMs a Low-Resource Language: Enhancing Code Completion in Pharo
Graz University of Technology(格拉茨工业大学);Software Institute – USI(软件研究所 - 瑞士意大利大学);Università della Svizzera italiana(瑞士意大利大学);Univ. Lille, Inria, CNRS, Centrale Lille and Lifeware SA(里尔大学、法国国家信息与自动化研究院、法国国家科学研究中心、里尔中央理工学院和Lifeware公司);CIRAD(法国农业研究国际中心);CNRS, University of Bordeaux(法国国家科学研究中心、波尔多大学);Bordeaux INP, LaBRI, UMR5800(波尔多理工学院、拉布里实验室)
Corrigendum and Supplement to "Improve Language Modelling for Code Completion through Learning General Token Repetition of Source Code (with Optimized Memory)"
Don't Use a Cannon to Kill a Fly: Lightweight Model Editing for LLMs to Correct Deprecated API Recommendations
Department of Computer Science, City University of Hong Kong(香港城市大学计算机系);The State Key Laboratory of Blockchain and Data Security, Zhejiang University(浙江大学区块链与数据安全国家重点实验室);Hangzhou High-Tech Zone (Binjiang) Institute of Blockchain and Data Security(杭州高新技术开发区(滨江)区块链与数据安全研究院);Midea Group(美的集团)
Synergizing Code Coverage and Gameplay Intent: Coverage-Aware Game Playtesting with LLM-Guided Reinforcement Learning
College of Computer and Information Science, Southwest University(西南大学计算机与信息科学学院);College of Science and Technology, Nihon University(日本立命馆大学科学技术学院);Shenzhen International Graduate School, Tsinghua University(清华大学深圳国际研究生院);Waseda Institute for Advanced Study, Waseda University(早稻田大学高级研究机构)
Seeing is Coding: On the Effectiveness of Vision Language Models in Code Understanding
Shanghai Jiao Tong University China;Hohai University China;Singapore Management University Singapore;Imperial College London United Kingdom;East China Normal University \& Shanghai Innovation Institute China;Chongqing University China;Shanghai Jiao Tong University;Hohai University;Singapore Management University;Imperial College London;East China Normal University \& Shanghai Innovation Institute;Chongqing University