SWE Context Bench: A Benchmark for Context Learning in Coding
Independent Researcher（独立研究员）；University of Oxford（牛津大学）；University of Edinburgh（爱丁堡大学）；Technical University of Munich（慕尼黑工业大学）；MemoraX AI；National University of Singapore（新加坡国立大学）；Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）；Carnegie Mellon University（卡内基梅隆大学）
Large language models are increasingly used as programming agents for repository level software engineering tasks. While recent benchmarks evaluate correctness in realistic codebases, they largely treat tasks as independent and do not assess whether agents can reuse previous experience or contexts across related problems. As a result, the ability of agents to accumulate, retrieve, and apply prior experience, as well as the efficiency gains from such reuse, remains difficult to measure. We introduce SWE-ContextBench, a benchmark designed to explicitly evaluate context reuse in programming agents. Built on SWE-Bench Lite, SWE-Bench Multilingual, and SWE-Bench Verified, SWE-ContextBench consists of 1,100 base tasks with 376 related tasks derived from real dependency and reference relationships among GitHub issues and pull requests. SWE-ContextBench groups base tasks and related tasks with shared context across 51 unique repositories and 9 programming languages. The benchmark evaluates agents along three complementary dimensions: prediction accuracy, time efficiency, and cost efficiency. Using SWE-ContextBench, we study multiple context reuse settings, including oracle guided and autonomous retrieval, as well as full execution trajectories and compact summaries. Our results show that correctly selected summarized context improves resolution accuracy and substantially reduces runtime and token cost, particularly on harder tasks. In contrast, unfiltered or incorrectly selected context provides limited or negative benefits. These findings highlight the importance of context representation and retrieval quality, and position SWE-ContextBench as a principled benchmark for studying context reuse in programming agents.
SWE-Chain: Benchmarking Coding Agents on Chained Release-Level Package Upgrades
The Chinese University of Hong Kong（香港中文大学）；Independent（独立）；ELLIS；Technical University of Darmstadt（达姆施塔特技术大学）；Johns Hopkins University（约翰霍普金斯大学）；Monash University（墨尔本大学）
A Unified Issue Resolution Benchmark for Requirement Clarification, Planning, and Code Generation for Coding Agents
Singapore Management University（新加坡管理大学）；Monash University Malaysia（马来西亚莫纳什大学）；Daegu Gyeongbuk Institute of Science and Technology (DGIST)（大邱庆北科学技术院）；The Chinese University of Hong Kong（香港中文大学）；North Carolina State University（北卡罗来纳州立大学）；City University of Hong Kong（香港城市大学）；The Hong Kong University of Science and Technology（香港科技大学）；The University of Hong Kong（香港大学）
SWE-Game: Can Coding Agents Build the Games We Want?
Shanghai Jiao Tong University（上海交通大学）；Zhongguancun Academy（中关村学院）；Shanghai Innovation Institute（上海创新研究院）；Shenzhen University（深圳大学）；Beijing University of Posts and Telecommunications（北京邮电大学）；Elbetech Technology（埃尔贝特科技）
ProjDevBench: Benchmarking AI Coding Agents on End-to-End Project Development
Shanghai Jiao Tong University, Shanghai, China（上海交通大学）；Beijing Institute of Technology, Beijing, China（北京理工大学）；University of California, Merced, CA, USA（加州大学默塞德分校）；Shanghai Innovation Institute, Shanghai, China（上海创新研究院）
SWE-MeM: Learning Adaptive Memory Management for Long-Horizon Coding Agents
The Chinese University of Hong Kong, China（香港中文大学）；Shanghai Jiao Tong University, China（上海交通大学）；Tsinghua University, China（清华大学）；ByteDance, China（字节跳动）
SWE-EVO: Benchmarking Coding Agents in Long-Horizon Software Evolution Scenarios
FPT Software AI Center（FPT软件人工智能中心）；School of Computing and Information Systems（计算与信息系统学院）；University of Melbourne（墨尔本大学）；Center of AI Research（人工智能研究中心）；VinUniversity（文大学）
SWE-Explore: Benchmarking How Coding Agents Explore Repositories
Shanghai Jiao Tong University（上海交通大学）；Xinjiang University（新疆大学）；University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；Independent Researcher（独立研究者）；The Chinese University of Hong Kong（香港中文大学）
