代码大模型 / AI 编程

CodeFuse-13B: A Pretrained Multi-lingual Code Large Language Model

Peng Di, Jianguo Li, Hang Yu, Wei Jiang, Wenting Cai, Yang Cao, Chaoyu Chen, Dajun Chen, Hongwei Chen, Liang Chen, Gang Fan, Jie Gong, Zi Gong, Wen Hu, Tingting Guo, Zhichao Lei, Ting Li, Zheng Li, Ming Liang, Cong Liao, Bingchang Liu, Jiachen Liu, Zhiwei Liu, Shaojun Lu, Min Shen, Guangpei Wang, Huan Wang, Zhi Wang, Zhaogui Xu, Jiawei Yang, Qing Ye, Gehao Zhang, Yu Zhang, Zelin Zhao, Xunjin Zheng, Hailian Zhou, Lifu Zhu, Xianying Zhu

专题命中 代码评测 ：code generation(abstract)；分类 cs.SE、cs.AI、cs.LG

Comments Accepted by ICSE-SEIP 2024

CRUXEval: A Benchmark for Code Reasoning, Understanding and Execution

Alex Gu, Baptiste Rozière, Hugh Leather, Armando Solar-Lezama, Gabriel Synnaeve, Sida I. Wang

专题命中 代码评测 ：code model(abstract)；分类 cs.SE、cs.AI、cs.LG

Comments 71 pages, 29 figures

Decoding Data Quality via Synthetic Corruptions: Embedding-guided Pruning of Code Data

Yu Yang, Aaditya K. Singh, Mostafa Elhoushi, Anas Mahmoud, Kushal Tirumala, Fabian Gloeckle, Baptiste Rozière, Carole-Jean Wu, Ari S. Morcos, Newsha Ardalani

专题命中 代码评测 ：code generation(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 12 pages, 4 figures, Oral Presentation at 3rd Workshop on Efficient Natural Language and Speech Processing (ENLSP-III), NeurIPS 2023

Large Language Models Illuminate a Progressive Pathway to Artificial Healthcare Assistant: A Review

Mingze Yuan, Peng Bao, Jiajia Yuan, Yunhao Shen, Zifan Chen, Yi Xie, Jie Zhao, Yang Chen, Li Zhang, Lin Shen, Bin Dong

专题命中 代码评测 ：repository(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 24 pages, 1 figure, 3 tables

InterCode: Standardizing and Benchmarking Interactive Coding with Execution Feedback

John Yang, Akshara Prabhakar, Karthik Narasimhan, Shunyu Yao

专题命中 代码评测 ：code generation(abstract)；分类 cs.SE、cs.CL、cs.LG

Comments Project site with code and data: https://intercode-benchmark.github.io

Spanish Pre-trained BERT Model and Evaluation Data

José Cañete, Gabriel Chaperon, Rodrigo Fuentes, Jou-Hui Ho, Hojin Kang, Jorge Pérez

专题命中 代码评测 ：repository(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Published as workshop paper at Practical ML for Developing Countries Workshop @ ICLR 2020

A Systematic Study and Comprehensive Evaluation of ChatGPT on Benchmark Datasets

Md Tahmid Rahman Laskar, M Saiful Bari, Mizanur Rahman, Md Amran Hossen Bhuiyan, Shafiq Joty, Jimmy Xiangji Huang

专题命中 代码评测 ：code generation(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Accepted by ACL 2023 Findings. The first three authors contributed equally

Parsel: Algorithmic Reasoning with Language Models by Composing Decompositions

Eric Zelikman, Qian Huang, Gabriel Poesia, Noah D. Goodman, Nick Haber

专题命中 代码评测 ：program synthesis(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments humaneval results, clarity

Measuring The Impact Of Programming Language Distribution

Gabriel Orlanski, Kefan Xiao, Xavier Garcia, Jeffrey Hui, Joshua Howland, Jonathan Malmaud, Jacob Austin, Rishabh Singh, Michele Catasta

专题命中 代码评测 ：code model(abstract)；分类 cs.CL、cs.LG、cs.PL

Comments Accepted to ICML 2023, Code and data release: https://github.com/google-research/babelcode

Fault-Aware Neural Code Rankers

Jeevana Priya Inala, Chenglong Wang, Mei Yang, Andres Codas, Mark Encarnación, Shuvendu K Lahiri, Madanlal Musuvathi, Jianfeng Gao

专题命中 代码评测 ：code generation(abstract)；分类 cs.SE、cs.AI、cs.PL

Comments In the proceedings of Advances in Neural Information Processing Systems, 2022

Towards Using Data-Influence Methods to Detect Noisy Samples in Source Code Corpora

Anh T. V. Dau, Thang Nguyen-Duc, Hoang Thanh-Tung, Nghi D. Q. Bui

专题命中 代码评测 ：code model(abstract)；分类 cs.SE、cs.AI、cs.PL

Comments The 37th IEEE/ACM International Conference on Automated Software Engineering

Depression Symptoms Modelling from Social Media Text: A Semi-supervised Learning Approach

Nawshad Farruque, Randy Goebel, Sudhakar Sivapalan, Osmar Zaiane

专题命中 代码评测 ：repository(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Title and relevant changes are made

C-Pack of IPAs: A C90 Program Benchmark of Introductory Programming Assignments

Pedro Orvalho, Mikoláš Janota, Vasco Manquinho

专题命中 代码评测 ：program repair(abstract)；分类 cs.SE、cs.AI、cs.PL

Comments 3 pages, 3 tables, 1 GitHub url: https://github.com/pmorvalho/C-Pack-IPAs

SapientML: Synthesizing Machine Learning Pipelines by Learning from Human-Written Solutions

Ripon K. Saha, Akira Ura, Sonal Mahajan, Chenguang Zhu, Linyi Li, Yang Hu, Hiroaki Yoshida, Sarfraz Khurshid, Mukul R. Prasad

专题命中 代码评测 ：program synthesis(abstract)；分类 cs.SE、cs.AI、cs.LG

Comments Accepted to the Technical Track of ICSE 2022

Curriculum-Meta Learning for Order-Robust Continual Relation Extraction

Tongtong Wu, Xuekai Li, Yuan-Fang Li, Reza Haffari, Guilin Qi, Yujin Zhu, Guoqiang Xu

专题命中 代码评测 ：repository(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Accepted by AAAI2021

WizardCoder: Empowering Code Large Language Models with Evol-Instruct

Ziyang Luo, Can Xu, Pu Zhao, Qingfeng Sun, Xiubo Geng, Wenxiang Hu, Chongyang Tao, Jing Ma, Qingwei Lin, Daxin Jiang

机构 * Microsoft（微软公司） ； Hong Kong Baptist University（香港 Baptist 大学） 纠错

专题命中 代码评测 ：code generation(abstract,comments)；分类 cs.CL、cs.AI

Comments Large Language model, Code Generation, Code LLMs.This paper has been accepted to ICLR 2024. Please cite the ICLR version

Journal ref The Twelfth International Conference on Learning Representations (ICLR 2024)

Can GPT-O1 Kill All Bugs? An Evaluation of GPT-Family LLMs on QuixBugs

Haichuan Hu, Ye Shang, Guolin Xu, Congqing He, Quanjun Zhang

专题命中 代码评测 ：program repair(abstract,comments)；分类 cs.SE、cs.AI

Comments Accepted to the 6th International Workshop on Automated Program Repair (APR 2025)

INSPECT: Intrinsic and Systematic Probing Evaluation for Code Transformers

Anjan Karmakar, Romain Robbes

专题命中 代码评测 ：code model(abstract,comments)；分类 cs.SE、cs.LG

Comments Accepted to IEEE Transactions on Software Engineering. Extension of our previous paper "What do pre-trained code models know about code?" (ASE 2021, arXiv:2108.11308). 21 pages

TokEval: A Tokenizer Evaluation Suite

TokEval：一个分词器评估套件

Clara Meister

机构 * EPFL（洛桑联邦理工学院） 纠错

专题命中 代码评测 ：code generation(abstract)；分类 cs.CL、cs.LG

AI总结 本研究推出TokEval框架，通过信息论、结构敏感等指标评估分词器，经实验验证其可预测下游模型性能，助力更原则性的分词器评估。

Comments Published as a conference paper at COLM 2026; Library hosted at https://github.com/cimeister/tokenizer-intrinsic-evals

Benchmarking Automated Security Patch Backporting: How Far Are We?

自动化安全补丁回移植基准测试：我们还差多远？

Jincheng Yang, Yulong Fu, Chengwei Liu, Lyuye Zhang, Fangyuan Zhang, Bingyang Ren, Yang Liu, Hui Li

专题命中 代码评测 ：repository(abstract)；分类 cs.SE、cs.AI

AI总结 该研究推出涵盖多场景的Porting Benchmark基准，评估五类安全补丁回移植工具，发现工具泛化能力差、复杂补丁性能骤降，明确根本原因并指出优化方向。

Comments 13 pages, 3 figures. Accepted at ASE 2026. Artifact: https://doi.org/10.5281/zenodo.21785770

H-VAEP and H-xT: Valuing Offensive On-the-Ball Actions in Handball by Estimating Probabilities

H-VAEP与H-xT：通过概率估计评估手球进攻中持球动作的价值

Julius Broermann, Oliver Müller, Michael Döring, Jochen Baumeister

机构 * Paderborn University（帕德博恩大学） ； SG Flensburg-Handewitt（弗伦斯堡-汉德维特体育俱乐部） 纠错

专题命中 代码评测 ：repository(abstract)；分类 cs.AI、cs.LG

AI总结 本文将足球的xT与VAEP框架适配至手球，开发H-xT与H-VAEP模型，利用手球德甲数据验证其有效性并发布代码，实现了手球球员的合理评估。

Comments 13 pages, 6 figures, 1 table. Accepted at the 13th Workshop on Machine Learning and Data Mining for Sports Analytics (MLSA 2026), co-located with ECML PKDD 2026

Aftab: A Comprehensive Benchmark of CNN Encoders and Advanced Value Functions in Parallelized Q-Networks

Aftab：并行Q网络中CNN编码器与高级价值函数的综合基准

Taha Shieenavaz, Shabnam Zareshahraki, Loris Nanni

机构 * University of Padua（帕多瓦大学） 纠错

专题命中 代码评测 ：repository(abstract)；分类 cs.AI、cs.LG

AI总结 本研究针对并行Q网络，设计评估8种CNN拓扑并集成多种Q学习扩展，提出复合架构Aftab，在Atari-57与Procgen Hard基准上均优于基线，已开源。

Towards the Harness of Embodied Agents

迈向具身智能体的管控

Qi Wang, Tianyi Wang, Chengyang Li, Shikun Ban, Yurun Chen, Yizhong Ge, Jason Qin, Chengtai Li, Wentao Zhu

专题命中 代码评测 ：coding agent(abstract)；分类 cs.AI、cs.LG

AI总结 本文提出名为Thea的具身智能体管控系统，通过场景图和退出码评估弥合物理世界与智能体的差距，实现长程任务完成。

Comments Project page: https://eit-hai.github.io/thea

Optimizing Expert-Designed Crystal Graph Networks for Band-Gap Prediction with an Autonomous LLM Research Loop

利用自主LLM研究循环优化专家设计的晶体图网络用于带隙预测

Chenmu Zhang, Boris I. Yakobson

机构 * Department of Materials Science and NanoEngineering（材料科学与纳米工程系） 纠错

专题命中 代码评测 ：coding agent(abstract)；分类 cs.AI、cs.LG

AI总结 提出一个自主LLM研究循环，在MatBench带隙基准上构建了无需外部预训练的最准确模型，超越了所有17个专家设计模型，通过实现元素对特征和空间群嵌入等已知方法。

Causal Agent based on Large Language Model

基于大语言模型的因果智能体

Kairong Han, Kun Kuang, Ziyu Zhao, Junjian Ye, Fei Wu

专题命中 代码评测 ：repository(abstract)；分类 cs.CL、cs.AI

AI总结 该研究针对LLM难以处理因果问题的挑战，提出Causal Agent框架，构建CausalTQA基准，实验显示其在多层级因果问题及真实数据集上性能优于SOTA。

Forensic Reproducibility Audit of a Radiology Vision-Language Model Benchmark: From Intended Protocol to Released Artifact

放射学视觉语言模型基准的法证可重复性审计：从预期协议到发布工件

Mateusz Kozłowski

专题命中 代码评测 ：repository(abstract)；分类 cs.CL、cs.AI

AI总结 对胸部X光视觉语言模型试点进行法证可重复性审计，追踪提示绑定等多方面情况，发现存在图像渲染、数据分割等问题，重建队列改变统计值，撤回原声明并指定机器可验证控制。

Comments Withdrawn by the author. On further review, the archived artifacts underlying this audit are too incomplete to support the reported statistics, and the paper's conclusions do not follow from the available evidence. The work is withdrawn in full; earlier versions should not be cited

InfiniteScienceGym: An Unbounded, Procedurally-Generated Benchmark for Scientific Analysis

InfiniteScienceGym：一个无界的、程序生成的科学分析基准

Oliver Bentham, Vivek Srikumar

机构 * Kahlert School of Computing（卡勒特计算学院） 纠错

专题命中 代码评测 ：repository(abstract)；分类 cs.CL、cs.AI

AI总结 本文提出InfiniteScienceGym，通过程序生成科学仓库和可验证问答任务，评估证据推理、回避和工具分析能力，发现现有模型在回答不可答问题上存在显著缺陷。

Comments 31 pages, 5 figures, 8 tables. Accepted to COLM 2026. See https://infinitesciencegym.github.io/

AI Evaluation Should Measure Verification Cost, Not Correctness Alone

AI评估应衡量验证成本，而非仅正确性

Viviana Crescitelli, Generoso Immediato, Fabio Persia, Stefania Costantini

机构 * Hitachi, Ltd.（日立制作所） ； Hitachi Rail（日立铁路） 纠错

专题命中 代码评测 ：code generation(abstract)；分类 cs.SE、cs.AI

AI总结 该研究指出AI评估仅靠正确性不足，提出需纳入验证成本，定义了验证成本错误，通过代码生成等证据说明高基准准确率可能掩盖高验证成本，主张评估应考虑现实资源约束下的错误可检测性。

Comments 20 pages, 1 table

ATLAS: Agentic Taxonomy of Large-Scale Software Ecosystems

ATLAS: 大规模软件生态系统的智能体分类体系

Junyi Lu, Mengyao Lyu, Jiahui Wu, Lei Yu, Chengwei Liu, Fengjun Zhang, Li Yang, Chun Zuo, Yang Liu

机构 * Institute of Software, Chinese Academy of Sciences（中国科学院软件研究所） ； University of Chinese Academy of Sciences（中国科学院大学） ； Nanyang Technological University（南洋理工大学） ； Nankai University（南开大学） ； Sinosoft Company Limited（中软国际有限公司） 纠错

专题命中 代码评测 ：repository(abstract)；分类 cs.SE、cs.CL

AI总结 提出ATLAS框架，结合LLM全局知识与实际仓库分布，通过智能体协作和自修正循环自动构建软件仓库的层次化分类体系，在54,387个GitHub仓库上评估，分类质量F1达83.13%，优于基线15个百分点。

Comments Accepted at the 41st IEEE/ACM International Conference on Automated Software Engineering (ASE 2026)

Beyond Questions: Evaluating LLM's Knowledge Expression

超越问题：评估大型语言模型（实际）知道什么

Luca Giordano, Simon Razniewski

机构 * ScaDS.AI Dresden/Leipzig & TU Dresden, Germany（ScaDS.AI 德尔布兰德/莱比锡及德累斯顿技术大学，德国） 纠错

专题命中 代码评测 ：repository(abstract)；分类 cs.CL、cs.AI

AI总结 提出开放知识评估新范式，通过开放式提示（如“告诉我关于M.L. King的一切”）评估模型自然表达的知识，并构建BeQu基准测试10,000个实体。