发表机构
Intel Corporation; Peking University（英特尔公司; 北京大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
ExecuCritic提出联合训练编码器与评论器的框架，利用执行反馈校准评论器，仅在评论器与执行器一致时指导编码器，在八个基准上优于基线，提升信用分配效率。
AI 中文摘要
执行反馈是代码模型的一种有用监督信号，因为单元测试是客观的且直接衡量程序正确性。其弱点在于，整个程序往往被简化为一个通过或失败的比特位，使得基于强化学习的验证（RLVR）面临困难的信用分配问题。与此同时，编码系统通常包含独立的审查者或测试者角色，但这些评论器通常是通过提示而非训练得到的，并且未针对执行结果进行校准。我们提出ExecuCritic，一种联合训练框架，其中编码器和评论器在同一执行轨迹上更新。评论器预测通过或失败的结果，并给出简短的诊断反馈；编码器仅在评论器与执行器在当前轨迹组上意见一致时使用该信号。在八个代码基准和两个近期开放骨干模型上，ExecuCritic相较于无评论器的GRPO、提示式审查系统以及标量奖励模型基线均有改进，同时需要更少的策略梯度步骤和更少的沙箱执行。消融研究和可靠性分析表明，性能提升源于更好的信用分配，而非更大的采样预算。
英文摘要
Execution feedback is a useful supervision signal for code models because unit tests are objective and directly measure program correctness. Its weakness is that an entire program is often reduced to one pass or fail bit, leaving RLVR to solve a difficult credit assignment problem. At the same time, coding systems often include separate reviewer or tester roles, but these critics are usually prompted rather than trained and are not calibrated against execution. We propose ExecuCritic, a joint training framework in which a coder and a critic are updated on the same execution rollouts. The critic predicts pass or fail outcomes and gives short diagnostic feedback; the coder uses this signal only when the critic agrees with the executor on the current rollout group. Across eight code benchmarks and two recent open backbones, ExecuCritic improves over GRPO without a critic, prompted reviewer systems and scalar reward model baselines, while requiring fewer policy gradient steps and fewer sandbox executions. Ablations and reliability analyses suggest that the gains come from better credit assignment rather than larger sampling budgets.
Comments33rd International Conference on Neural Information Processing (ICONIP 2026)
