基于执行的信用分配用于代码生成中的GRPO
Execution-Grounded Credit Assignment for GRPO in Code Generation
AI总结：
本文提出基于执行的信用分配方法EGCA，通过执行轨迹局部化GRPO更新，提升代码生成的单元测试通过率，实验表明在HumanEval和MBPP数据集上效果优于GRPO。
摘要：
无批评者强化学习结合可验证奖励（RLVR）通过优化单元测试通过率提升代码生成，但GRPO风格更新面临粗略的信用分配问题：单个结果信号在长程序中均匀分配，即使失败源于局部语义错误。本文提出基于执行的信用分配（EGCA），通过执行轨迹局部化GRPO更新。对于满足算法约束但测试失败的程序，EGCA在相同仪器下执行候选程序和标准参考解决方案（离线 curated；用于分析而非监督），识别最早语义分歧，仅对相应token跨度分配优势并屏蔽后续token。EGCA是一种无需批评者、辅助损失或学习验证者的直接修改，实现82.1%的HumanEval pass@1（比GRPO高3.1%）和68.9%的MBPP，仅增加18%的运行时间开销。
英文摘要：
Critic-free reinforcement learning with verifiable rewards (RLVR) improves code generation by optimizing unit-test pass rates, but GRPO-style updates suffer from coarse credit assignment: a single outcome signal is spread uniformly across long programs even when failure stems from a localized semantic error. We propose Execution-Grounded Credit Assignment (EGCA), which localizes GRPO updates using execution traces. For programs that satisfy algorithmic constraints but fail tests, EGCA executes the candidate and a canonical reference solution (curated once offline; used for analysis, not supervision) under identical instrumentation, identifies the earliest semantic divergence, and assigns advantage only to the corresponding token span while masking downstream tokens. EGCA is a drop-in modification requiring no critic, auxiliary loss, or learned verifier, yielding 82.1% pass@1 on HumanEval (+3.1 over GRPO) and 68.9% on MBPP (+1.5) with 18% wall-clock overhead.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
