In this paper, we present a comparative study of five autonomous coding agents using AIDev-pop, which is a public dataset containing thousands of AI-generated pull requests (PRs) across popular open-source repositories. We evaluate agents' performance along three task-aware dimensions spanning the PR lifecycle: (1) PR acceptance rate, (2) review discussion volume, and (3) commit message quality. Our quantitative analysis finds that Codex consistently achieves high PR acceptance rates across most task categories, while Copilot's PRs trigger the highest volume of both human and automated review discussions. In contrast, commit-level quality varies independently of acceptance outcomes. Claude and Cursor produce higher proportions of high-quality commit messages across several task types, and Codex exhibiting comparatively lower commit quality despite strong integration outcomes. Our findings inform selection and improvements of AI agents for their effective integration to collaborative software engineering.
ProjDevBench: Benchmarking AI Coding Agents on End-to-End Project Development
Shanghai Jiao Tong University, Shanghai, China（上海交通大学）；Beijing Institute of Technology, Beijing, China（北京理工大学）；University of California, Merced, CA, USA（加州大学默塞德分校）；Shanghai Innovation Institute, Shanghai, China（上海创新研究院）
SABER: Benchmarking Operational Safety of LLM Coding Agents in Stateful Project Workspaces
The University of Hong Kong（香港大学）；Shandong University（山东大学）；Carnegie Mellon University（卡内基梅隆大学）；National University of Singapore（新加坡国立大学）；The Hong Kong University of Science and Technology（香港科技大学）
Making Agent-Mediated Contributions Governable: A Project-Level Governance Manifest for Open-Source AI Collaboration
School of Business Administration, Shanxi University of Finance and Economics（山西财经大学工商管理学院）；Experiment and Practical Training Center, Shanxi University of Finance and Economics（山西财经大学实验实训中心）；School of Computer Science and Technology, North University of China（中北大学计算机科学与技术学院）；Department of Computer Science, University of Warwick（华威大学计算机科学系）；School of Information, Shanxi University of Finance and Economics（山西财经大学信息学院）
Using Process Mining to Generate AI Agents from Software Engineering Process Records
Hasso Plattner Institute (HPI), University of Potsdam（波茨坦大学哈索·普拉特纳研究院（HPI））；IBM Software Innovation Lab (SIL)（IBM软件创新实验室（SIL））；University of Haifa（海法大学）；paluno Institute, University of Duisburg Essen（杜伊斯堡-埃森大学paluno研究院）
