SWE-rebench V2: 大规模语言无关的SWE任务集合
SWE-rebench V2: Language-Agnostic SWE Task Collection at Scale
AI总结：
提出语言无关的自动化流水线SWE-rebench V2，用于大规模收集可执行的软件工程任务并构建RL训练环境，产出涵盖20种语言、3617个仓库的32079个任务数据集。
摘要：
软件工程智能体（SWE）正在快速进步，最近的进展主要由强化学习（RL）驱动。然而，RL训练受到大规模任务集合稀缺性的限制，这些任务需要具有可复现的执行环境和可靠的测试套件。尽管越来越多的基准测试出现，适合训练的数据集在规模和多样性上仍然有限，或者通常针对有限的高资源语言生态系统。我们引入SWE-rebench V2，一个语言无关的自动化流水线，用于大规模收集可执行的真实世界SWE任务并构建RL训练环境。该流水线通过交互式设置智能体合成仓库特定的安装和测试程序，并使用一组LLM评判器过滤不合理的实例，这些评判器经过人工验证的SWE-bench注释验证。使用该流水线，我们构建了一个包含20种语言、3617个仓库的32079个任务数据集，并附带预构建的镜像以实现可复现执行。为了进一步扩展训练数据，我们还发布了超过120000个任务，包含安装说明、失败到通过的测试和丰富的元数据，其中问题陈述基于原始拉取请求描述生成。我们通过一项诊断研究验证了收集的实例，该研究涵盖了五种编程语言中的一部分任务，涉及七个流行模型，并提供了实例级元数据，标记了常见的混淆因素，如过于严格的测试和描述不充分。我们发布了数据集、收集和执行代码以及相关工件，以支持跨多种语言和仓库的大规模SWE智能体训练。
英文摘要：
Software engineering agents (SWE) are improving rapidly, with recent gains largely driven by reinforcement learning (RL). However, RL training is constrained by the scarcity of large-scale task collections with reproducible execution environments and reliable test suites. Although a growing number of benchmarks have emerged, datasets suitable for training remain limited in scale and diversity or often target a limited set of high-resource language ecosystems. We introduce SWE-rebench V2, a language-agnostic automated pipeline for harvesting executable real-world SWE tasks and constructing RL training environments at scale. The pipeline synthesizes repository-specific installation and test procedures via an interactive setup agent, and filters unsound instances using an ensemble of LLM judges, validated against human-verified SWE-bench annotations. Using this pipeline, we construct a dataset of 32,079 tasks spanning 20 languages and 3,617 repositories, with pre-built images for reproducible execution. To further scale training data, we additionally release 120,000+ tasks with installation instructions, fail-to-pass tests and rich metadata, where the problem statement is generated based on the original pull request description. We validate the collected instances through a diagnostic study that covers a subset of tasks in five programming languages across seven popular models, and provide instance-level metadata that flags common confounders such as overly restrictive tests and underspecified descriptions. We release the datasets, the collection and execution code, and associated artifacts to enable large-scale training of SWE agents across diverse languages and repositories.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
