代码大模型 / AI 编程

Ensuring Reproducibility in Generative AI Systems for General Use Cases: A Framework for Regression Testing and Open Datasets

Masumi Morishige, Ryo Koshihara

机构 * Galirage Inc.（加利拉格公司） 纠错

专题命中 代码评测 ：code generation(abstract)；分类 cs.CL、cs.AI

Comments 15 pages, 10 figures

CoCo-Bench: A Comprehensive Code Benchmark For Multi-task Large Language Model Evaluation

Wenjing Yin, Tianze Sun, Yijiong Yu, Jiawei Fang, Guangyao Su, Jiancheng Wang, Zekun Wang, Wei Wang, Ran Chen, Ziyun Dai, Shuai Yuan, Menghang Dong, Peng Luo, Dong Cao, Da Lei, Yajun Zhang, Hao Chen, Xiang Ma, Yong Liu, Weifeng Liu, Yuanjian Xu, Ji Pei

专题命中 代码评测 ：code generation(abstract)；分类 cs.SE、cs.AI

Comments Submitted to ACL 2025. Under review

Large Language Models are Qualified Benchmark Builders: Rebuilding Pre-Training Datasets for Advancing Code Intelligence Tasks

Kang Yang, Xinjun Mao, Shangwen Wang, Yanlin Wang, Tanghaoran Zhang, Bo Lin, Yihao Qin, Zhang Zhang, Yao Lu, Kamal Al-Sabahi

机构 * College of Computer, National University of Defense Technology（计算机学院，国防科技大学） ； Sun Yat-sen University（中山大学） ； College of Banking and Financial Studies（金融学院） 纠错

专题命中 代码评测 ：code model(abstract)；分类 cs.SE、cs.CL

Comments Awarded the ACM SIGSOFT Distinguished Paper Award in ICPC 2025

Gradient-Optimized Fuzzy Classifier: A Benchmark Study Against State-of-the-Art Models

Magnus Sieverding, Nathan Steffen, Kelly Cohen

机构 * University of Cincinnati（辛辛那提大学） 纠错

专题命中 代码评测 ：repository(abstract)；分类 cs.AI、cs.LG

Can LLMs Replace Human Evaluators? An Empirical Study of LLM-as-a-Judge in Software Engineering

Ruiqi Wang, Jiyu Guo, Cuiyun Gao, Guodong Fan, Chun Yong Chong, Xin Xia

机构 * Harbin Institute of Technology, Shenzhen（哈尔滨工业大学（深圳）） ； Monash University Malaysia（墨尔本大学马来西亚分校） ； Zhejiang University（浙江大学） 纠错

专题命中 代码评测 ：code generation(abstract)；分类 cs.SE、cs.AI

Comments Accepted by ISSTA 2025: https://conf.researchr.org/details/issta-2025/issta-2025-papers/85/Can-LLMs-replace-Human-Evaluators-An-Empirical-Study-of-LLM-as-a-Judge-in-Software-E

RoboTwin: Dual-Arm Robot Benchmark with Generative Digital Twins

Yao Mu, Tianxing Chen, Zanxin Chen, Shijia Peng, Zhiqian Lan, Zeyu Gao, Zhixuan Liang, Qiaojun Yu, Yude Zou, Mingkun Xu, Lunkai Lin, Zhiqiang Xie, Mingyu Ding, Ping Luo

专题命中 代码评测 ：code generation(abstract)；分类 cs.CL、cs.AI

Comments CVPR 2025 Highlight. 22 pages. Project page: https://robotwin-benchmark.github.io/

RoboTwin: Dual-Arm Robot Benchmark with Generative Digital Twins (early version)

Yao Mu, Tianxing Chen, Shijia Peng, Zanxin Chen, Zeyu Gao, Yude Zou, Lunkai Lin, Zhiqiang Xie, Ping Luo

专题命中 代码评测 ：code generation(abstract)；分类 cs.CL、cs.AI

Comments Project page: https://robotwin-benchmark.github.io/early-version/

OpenTuringBench: An Open-Model-based Benchmark and Framework for Machine-Generated Text Detection and Attribution

Lucio La Cava, Andrea Tagarelli

专题命中 代码评测 ：repository(abstract)；分类 cs.CL、cs.AI

Comments Under review with ARR

Agent-Arena: A General Framework for Evaluating Control Algorithms

Halid Abdulrahim Kadi, Kasim Terzić

专题命中 代码评测 ：repository(abstract)；分类 cs.SE、cs.AI

Comments 20 pages and 1 figure

Your Image Generator Is Your New Private Dataset

Nicolo Resmini, Eugenio Lomurno, Cristian Sbrolli, Matteo Matteucci

专题命中 代码评测 ：repository(abstract)；分类 cs.AI、cs.LG

Uncertainty Propagation in XAI: A Comparison of Analytical and Empirical Estimators

Teodor Chiaburu, Felix Bießmann, Frank Haußer

专题命中 代码评测 ：repository(abstract)；分类 cs.AI、cs.LG

Comments 23 pages, 10 figures, accepted at WCXAI 2025 Istanbul

FAIRE: Assessing Racial and Gender Bias in AI-Driven Resume Evaluations

Athena Wen, Tanush Patil, Ansh Saxena, Yicheng Fu, Sean O'Brien, Kevin Zhu

专题命中 代码评测 ：repository(abstract)；分类 cs.CL、cs.AI

TS-Inverse: A Gradient Inversion Attack Tailored for Federated Time Series Forecasting Models

Caspar Meijer, Jiyue Huang, Shreshtha Sharma, Elena Lazovik, Lydia Y. Chen

专题命中 代码评测 ：repository(abstract)；分类 cs.AI、cs.LG

EnvBench: A Benchmark for Automated Environment Setup

Aleksandra Eliseeva, Alexander Kovrigin, Ilia Kholkin, Egor Bogomolov, Yaroslav Zharov

专题命中 代码评测 ：repository(abstract)；分类 cs.SE、cs.LG

Comments Accepted at the DL4Code workshop at ICLR'25

Job Shop Scheduling Benchmark: Environments and Instances for Learning and Non-learning Methods

Robbert Reijnen, Igor G. Smit, Hongxiang Zhang, Yaoxin Wu, Zaharah Bukhsh, Yingqian Zhang

专题命中 代码评测 ：repository(abstract)；分类 cs.AI、cs.LG

A Survey of Sim-to-Real Methods in RL: Progress, Prospects and Challenges with Foundation Models

Longchao Da, Justin Turnau, Thirulogasankar Pranav Kutralingam, Alvaro Velasquez, Paulo Shakarian, Hua Wei

专题命中 代码评测 ：repository(abstract)；分类 cs.AI、cs.LG

Comments 19 pages, 6 figures, 5 tables

Keeping Yourself is Important in Downstream Tuning Multimodal Large Language Model

Wenke Huang, Jian Liang, Xianda Guo, Yiyang Fang, Guancheng Wan, Xuankun Rong, Chi Wen, Zekun Shi, Qingyun Li, Didi Zhu, Yanbiao Ma, Ke Liang, Bin Yang, He Li, Jiawei Shao, Mang Ye, Bo Du

专题命中 代码评测 ：repository(abstract)；分类 cs.CL、cs.AI

BIOSCAN-5M: A Multimodal Dataset for Insect Biodiversity

Zahra Gharaee, Scott C. Lowe, ZeMing Gong, Pablo Millan Arias, Nicholas Pellegrino, Austin T. Wang, Joakim Bruslund Haurum, Iuliia Zarubiieva, Lila Kari, Dirk Steinke, Graham W. Taylor, Paul Fieguth, Angel X. Chang

专题命中 代码评测 ：repository(abstract)；分类 cs.AI、cs.LG

Journal ref NeurIPS 2024

Exploring Code Language Models for Automated HLS-based Hardware Generation: Benchmark, Infrastructure and Analysis

Jiahao Gai, Hao Mark Chen, Zhican Wang, Hongyu Zhou, Wanru Zhao, Nicholas Lane, Hongxiang Fan

专题命中 代码评测 ：code generation(abstract)；分类 cs.SE、cs.LG

Comments Paper accepted by ASP-DAC'25

CoCoNUT: Structural Code Understanding does not fall out of a tree

Claas Beger, Saikat Dutta

专题命中 代码评测 ：code generation(abstract)；分类 cs.SE、cs.LG

Comments Accepted at 2025 IEEE/ACM International Workshop on Large Language Models for Code (LLM4Code)

From Matching to Generation: A Survey on Generative Information Retrieval

Xiaoxi Li, Jiajie Jin, Yujia Zhou, Yuyao Zhang, Peitian Zhang, Yutao Zhu, Zhicheng Dou

专题命中 代码评测 ：repository(abstract)；分类 cs.CL、cs.AI

Performance Review on LLM for solving leetcode problems

Lun Wang, Chuanqi Shi, Shaoshui Du, Yiyi Tao, Yixian Shen, Hang Zheng, Yanxin Shen, Xinyu Qiu

专题命中 代码评测 ：code generation(abstract)；分类 cs.SE、cs.AI

A Survey on Large Language Model based Autonomous Agents

Lei Wang, Chen Ma, Xueyang Feng, Zeyu Zhang, Hao Yang, Jingsen Zhang, Zhiyuan Chen, Jiakai Tang, Xu Chen, Yankai Lin, Wayne Xin Zhao, Zhewei Wei, Ji-Rong Wen

专题命中 代码评测 ：repository(abstract)；分类 cs.CL、cs.AI

Comments Correcting several typos, 35 pages, 5 figures, 3 tables

Multimodal Learning for Just-In-Time Software Defect Prediction in Autonomous Driving Systems

Faisal Mohammad, Duksan Ryu

专题命中 代码评测 ：repository(abstract)；分类 cs.SE、cs.AI

Comments 9

Towards consistency of rule-based explainer and black box model -- fusion of rule induction and XAI-based feature importance

Michał Kozielski, Marek Sikora, Łukasz Wawrowski

专题命中 代码评测 ：repository(abstract)；分类 cs.AI、cs.LG

Show, Don't Tell: Evaluating Large Language Models Beyond Textual Understanding with ChildPlay

Gonçalo Hora de Carvalho, Oscar Knap, Robert Pollice

专题命中 代码评测 ：repository(abstract)；分类 cs.CL、cs.AI

An Extensive Evaluation of PDDL Capabilities in off-the-shelf LLMs

Kaustubh Vyas, Damien Graux, Sébastien Montella, Pavlos Vougiouklis, Ruofei Lai, Keshuang Li, Yang Ren, Jeff Z. Pan

专题命中 代码评测 ：code generation(abstract)；分类 cs.CL、cs.AI

Comments Under review

Isolating Language-Coding from Problem-Solving: Benchmarking LLMs with PseudoEval

Jiarong Wu, Songqiang Chen, Jialun Cao, Hau Ching Lo, Shing-Chi Cheung

专题命中 代码评测 ：code generation(abstract)；分类 cs.SE、cs.CL

Code Summarization Beyond Function Level

Vladimir Makharev, Vladimir Ivanov

专题命中 代码评测 ：repository(abstract)；分类 cs.CL、cs.AI

Comments Accepted to LLM4Code @ ICSE'25; 8 pages, 3 figures, 4 tables

TritonBench: Benchmarking Large Language Model Capabilities for Generating Triton Operators

Jianling Li, Shangzhan Li, Zhenye Gao, Qi Shi, Yuxuan Li, Zefan Wang, Jiacheng Huang, Haojie Wang, Jianrong Wang, Xu Han, Zhiyuan Liu, Maosong Sun

专题命中 代码评测 ：code generation(abstract)；分类 cs.CL、cs.LG