CodeWalk：通过遍历问题图生成编码基准
CodeWalk: Generating Coding Benchmarks by Walking a Problem Graph
AI总结：
本研究提出CodeWalk方法，通过程序化组合现有原子问题生成编码基准，无需语言模型生成验证，生成的任务可提升Qwen3.5-9B性能且对前沿模型具挑战性。
摘要：
高质量编码问题的收集成本高昂，SWE-bench等基准依赖大量人工整理，而合成替代方案通常需要代价高昂的模型生成与验证。本研究表明，通过组合现有问题几乎可以零成本创建新编码问题。我们将问题生成形式化为代码组合：将经过验证的原子问题组合为N级任务，仅当每个原子问题均未被解决且联合参考解通过评估时使用该任务。整个任务构建过程是程序化的，无需语言模型生成或验证。我们在四个编码场景中实例化该方法：多错误修复、证明补全、复杂度分析与克隆等价性。生成的任务对前沿模型具有挑战性，所有模型均随组合深度增加而性能下降；生成的任务也可用作训练数据，在其上进行拒绝采样微调可在所有组合级别提升Qwen3.5-9B的性能。
英文摘要：
High-quality coding problems are expensive to collect. Benchmarks such as SWE-bench rely on substantial human curation, while synthetic alternatives typically rely on costly model generation and validation. In this work, we show that new coding problems can be created at almost no cost by composing existing ones. We formulate problem generation as code composition: verified atomic problems are combined into level-$N$ tasks, and a task is kept only if every atomic problem remains unsolved and a joint reference solution passes the same check used at evaluation time. The entire task-construction process is programmatic and requires no language-model generation or validation. We instantiate this approach in four coding settings: multi-bug repair, proof completion, complexity analysis, and clone equivalence. The generated tasks are challenging for frontier models, which all degrade as compositional depth increases. The generated tasks are also useful as training data: rejection-sampling fine-tuning on them improves Qwen3.5-9B at every composition level.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
