从片段到轨迹:软件工程代理的特权过程监督
From Patches to Trajectories: Privileged Process Supervision for Software-Engineering Agents
AI总结:
本文提出Patches-to-Trajectories (P2T)方法,通过利用开发者撰写的参考补丁来改进软件工程代理的训练过程,提高训练效果和效率。
摘要:
监督微调(SFT)在长教师轨迹上的应用是使开放软件工程(SWE)代理具备调查和推理能力的主要方法。由于每个保留的响应都成为模仿目标,学生继承了最终结果和中间缺陷,包括无根据的跳跃和冗余循环。高质量的训练数据必须有效(每一步都基于事实并缩小代理的知识差距到正确修复)且高效(每一步都有信息而非冗余或循环)。现有方法仅使用二进制终端验证器过滤或重新标记教师轨迹,这并未直接针对这些方面,也无法对教师失败的实例提供监督。大多数真实问题包含一个开发者撰写的参考补丁,$p^\star$,揭示了正确修复所假设的文件路径、运行时行为和编码规范,但标准流程却将其丢弃。我们提出Patches-to-Trajectories(P2T),在整理过程中使用$p^\star$作为特权信息,并将轨迹构建制定为对每一步有效性与轨迹长度的双目标优化。一个反向阶段将$p^\star$转化为一个上下文事实和解决方案里程碑的潜在过程图,$G^\star$。一个正向阶段通过在泄漏阻断的 groundedness 检查下对每一步进展进行评分,从盲目的教师延续中整理轨迹,并保留最短的有效段。仅使用1.8k整理的SWE-Gym实例,P2T在结果过滤SFT及其工具错误掩码变体上提高了效果和效率。在SWE-bench Verified上,它将Pass@1提高多达10.8个点,同时将每实例推理成本减少约15%,在SWE-bench Lite上也保持一致的收益。大小匹配的消融分析和定性分析进一步将轨迹质量与数据规模分离。
英文摘要:
Supervised fine-tuning (SFT) on long teacher trajectories is the dominant way to instill investigation and reasoning in open software-engineering (SWE) agents. Since every retained response becomes an imitation target, the student inherits the final outcome and intermediate flaws, including ungrounded leaps and redundant loops. High-quality training data must be effective(each step is grounded and narrows the agent's epistemic gap to the correct fix) and efficient(each step is information-bearing rather than redundant or looping). Existing recipes filter or relabel teacher rollouts using only a binary terminal verifier, which does not directly target these axes and provides no supervision on instances where the teacher fails. Most real issue includes a developer-authored reference patch, $p^\star$, revealing the file paths, runtime behaviors, and coding conventions presupposed by the correct fix, yet standard pipelines discard it. We propose Patches-to-Trajectories (P2T), which uses $p^\star$ as privileged information during curation and formulates trajectory construction as bi-objective optimization over per-step effectiveness and trajectory length. A reverse phase distills $p^\star$ into a latent process graph, $G^\star$, of contextual facts and solution milestones. A forward phase curates trajectories from blinded teacher continuations by scoring per-step progress against $G^\star$ under a leakage-blocking groundedness check and retaining the shortest effective segments. Using only 1.8k curated SWE-Gym instances, P2T improves effectiveness and efficiency over outcome-filtered SFT and its tool-error-masking variant. On SWE-bench Verified, it raises Pass@1 by up to 10.8 points while reducing per-instance inference cost by ~15%, with consistent gains on SWE-bench Lite. Size-matched ablations and qualitative analysis further isolate trajectory quality from data scale.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。