Repository-level code agents have shown strong promise in real-world feature addition tasks, making reliable evaluation of their capabilities increasingly important. However, existing benchmarks primarily evaluate these agents as black boxes based on final test correctness, providing limited insight into how they reason and where failures arise. To address this limitation, we introduce RACE-bench, a reasoning-augmented benchmark for evaluating code agents on repository-level feature addition tasks. RACE-bench contains 528 real-world feature addition instances from 12 open-source repositories. Each instance is paired with executable patch verification and structured intermediate reference reasoning covering issue understanding, file localization, implementation tasks, and step decomposition. Based on this design, we introduce a dual-track evaluation framework that jointly measures patch correctness and intermediate reasoning alignment with developer-accepted reference trajectories. We evaluate three representative repository-level code agents on RACE-bench. On the full benchmark, Resolved Rate ranges from 29% to 70% across different agents. Our reasoning-level analysis further shows that while current agents perform well at understanding high-level intent, their performance degrades substantially when translating intent into concrete implementation steps. We also find patches that can be applied but still fail the tests cover fewer reference-reasoning elements (35.7% lower recall) and contain more unsupported reasoning elements (94.1% higher over-prediction) than successful patches. These findings highlight the importance of evaluating repository-level code agents beyond final patch correctness by examining the quality of their reasoning processes.
VITAL-RAG: Invariance Race for Context Allocation in Coding Agents
Nanjing University of Posts and Telecommunications（南京邮电大学）；Tianjin University（天津大学）；Institute for AI Industry Research, Tsinghua University（清华大学人工智能产业研究院）
A Scalable Benchmark for Repository-Oriented Long-Horizon Conversational Context Management
State Key Laboratory of Complex & Critical Software Environment, School of Computer Science and Engineering, Beihang University（北京航空航天大学计算机科学与工程学院复杂与关键软件环境国家重点实验室）
DependEval: Benchmarking LLMs for Repository Dependency Understanding
Nanyang Technological University（南洋理工大学）；Tsinghua University（清华大学）；Beihang University（北京航空航天大学）；ShanghaiTech University（上海科技大学）；The University of Hong Kong（香港大学）
CodeMEM: AST-Guided Adaptive Memory for Repository-Level Iterative Code Generation
School of Computer Science and Engineering, Beihang University（北京航空航天大学计算机学院）；School of Computing and Data Science, The University of Hong Kong（香港大学计算与数据科学学院）
FEA-Bench: A Benchmark for Evaluating Repository-Level Code Generation for Feature Implementation
State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University（信息处理国家重点实验室，计算机学院，北京大学）；Microsoft Research Asia（微软亚洲研究院）
Evaluating and Improving Automated Repository-Level Rust Issue Resolution with LLM-based Agents
Research Institute of Trustworthy Autonomous Systems, Southern University of Science and Technology（南方科技大学可信自主系统研究院）；Southern University of Science and Technology（南方科技大学）；Ant Group（蚂蚁集团）
Beyond Repository Boundaries: Cross-Repository Graph Retrieval for Code Generation
Quantum AI & Cyber Security Institute, FPT Corporation（量子人工智能与网络安全研究所，FPT公司）；Hanoi University of Science and Technology（河内科学技术大学）；Rutgers University（罗格斯大学）；The University of Melbourne（墨尔本大学）；Center for AI Research, VinUniversity（VinUniversity人工智能研究中心）
Prometheus: Towards Long-Horizon Codebase Navigation for Repository-Level Problem Solving
University College London（伦敦大学学院）；Sana Labs（Sana实验室）；Uppsala University（乌普萨拉大学）；Nanjing University（南京大学）；Carnegie Mellon University（卡内基梅隆大学）；KTH Royal Institute of Technology（瑞典皇家理工学院）
VibeMemBench: Evaluating Memory Systems for Coding Agents on Real Repository Coding Tasks
Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）；SUAT；Alibaba Token Hub, Alibaba Group（阿里巴巴集团阿里巴巴令牌中心）；Alibaba Group（阿里巴巴集团）
GREPO: A Benchmark for Graph Neural Networks on Repository-Level Bug Localization
Institute for Artificial Intelligence, Peking University（人工智能研究院，北京大学）；School of Intelligence Science and Technology, Peking University（智能科学与技术学院，北京大学）；College of Intelligence Science and Technology, National University of Defense Technology（智能科学与技术学院，国防科技大学）
EXPEREPAIR: Dual-Memory Enhanced LLM-based Repository-Level Program Repair
University of Chinese Academy of Sciences（中国科学院大学）；State Key Laboratory of Intelligent Game, Institute of Software, CAS（智能游戏国家重点实验室，软件研究所，中国科学院）；Beihang University（北京航空航天大学）；York University（约克大学）
