SWE-Chain：基于链式发布级包升级的编码代理基准测试
SWE-Chain: Benchmarking Coding Agents on Chained Release-Level Package Upgrades
AI总结：
SWE-Chain通过链式发布级包升级评估编码代理，包含12个升级链和155个版本过渡，揭示当前代理在连续维护中的不足。
摘要：
基于大语言模型的编码代理被期望执行更真实的软件维护任务。现有基准测试转向真实软件演进，但很少捕捉连续维护在发布级包升级的粒度，其中变更被打包、发布并继承至后续版本。本文提出SWE-Chain基准测试，评估代理在链式发布级包升级中的能力。通过设计分治合成管道，将发布说明与代码差异对齐，确保要求基于实际代码变更，对代理具有信息量且可实现。SWE-Chain包含9个真实Python包的12个升级链，155个版本过渡和1,660个基于实际变更的升级要求。在九种前沿代理模型配置下，代理在Build+Fix模式下平均达到44.8%的解决率、65.4%的精度和50.2%的F1分数，其中Claude-Opus-4.7（Claude Code）达到60.8%的解决率、80.6%的精度和68.5%的F1分数。这些结果表明SWE-Chain既可行又具有区分性，揭示当前代理在连续包发布升级中仍难以正确执行升级而不破坏现有功能。
英文摘要：
Coding agents powered by large language models are increasingly expected to perform realistic software maintenance tasks beyond isolated issue resolution. Existing benchmarks have shifted toward realistic software evolution, but they rarely capture continuous maintenance at the granularity of package releases, where changes are bundled, shipped, and inherited by subsequent versions. We present SWE-Chain, a benchmark for evaluating agents on chained release-level package upgrades, where each transition builds on the agent's prior codebase. To produce upgrade specifications, we design a divide-and-conquer synthesis pipeline that aligns release notes with code diffs for each version transition, ensuring the requirements are grounded in actual code changes, informative to agents, and feasible to implement. SWE-Chain contains 12 upgrade chains across 9 real Python packages, with 155 version transitions and 1,660 grounded upgrade requirements. Across nine frontier agent-model configurations, agents achieve an average of 44.8% resolving, 65.4% precision, and 50.2% F1 under the Build+Fix regime, with Claude-Opus-4.7 (Claude Code) leading at 60.8% resolving, 80.6% precision, and 68.5% F1. These results show that SWE-Chain is both feasible and discriminative, and reveal that current agents still struggle to make correct upgrades across chained package releases without breaking existing functionality.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
