通过执行进行推理：统一代码生成的过程奖励与结果奖励
Reasoning Through Execution: Unifying Process and Outcome Rewards for Code Generation
AI总结：
本文提出 ORPS，利用可执行验证、树搜索与融合运行时反馈的自我批判评分统一过程和结果监督，在多模型多基准上显著提升代码正确性与效率。
摘要：
大型语言模型擅长代码生成，但在处理需要复杂推理的编程任务时仍存在困难。为弥合这一差距，传统过程监督依赖学习得到的奖励模型，需要昂贵的训练数据，并受到奖励错位影响；而结果监督在需要协调中间步骤的复杂任务上会失效。我们提出 Outcome Refining Process Supervision（ORPS，结果精炼过程监督），通过可执行验证统一过程监督与结果监督：一个树结构搜索框架生成策略性备选方案，分析执行指标，并通过自我批判机制对候选方案评分，该机制将运行时反馈与推理相结合。在 5 个模型和 3 个基准上的实验显示出一致收益，正确性提高 26.9%，代码效率提升 42.2%。结果表明，ORPS 使 LLM 能够克服代码生成中的局部最优，为结合可验证结果与结构化推理以应对复杂挑战提供了有前景的方向。我们在 https://github.com/zhuohaoyu/ORPS 开源。
英文摘要：
Large Language Models excel at code generation yet struggle with complex programming tasks that demand sophisticated reasoning. To bridge this gap, traditional process supervision relies on learned reward models requiring costly training data and suffering from reward misalignment, while outcome supervision fails for complex tasks needing coordinated intermediate steps. We introduce Outcome Refining Process Supervision, which unifies process and outcome supervision by leveraging executable verification: a tree-structured search framework generates strategic alternatives, profiles execution metrics, and scores candidates via self-critique mechanisms that integrate runtime feedback with reasoning. Experiments across 5 models and 3 benchmarks show consistent gains, with 26.9% higher correctness and 42.2% improved code efficiency. The results demonstrate that ORPS enables LLMs to overcome local optima in code generation, suggesting a promising direction for combining verifiable outcomes with structured reasoning to tackle complex challenges. We open-source at: https://github.com/zhuohaoyu/ORPS
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
