arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2605.08680cs.SEcs.AIcs.LG
收藏

语义投票:基于执行的共识机制用于大语言模型代码生成

Semantic Voting: Execution-Grounded Consensus for LLM Code Generation

The University of Texas at Austin(德克萨斯大学奥斯汀分校)

Shan Jiang, Zijian Yi, Chenguang Zhu

AI总结:

本文研究了大语言模型代码生成中基于执行的共识机制,通过比较不同方法发现输入质量对结果影响更大,语义投票在多数配置中表现最佳。

摘要:

LLM代码生成管道通常生成多个候选并选择最终答案,但缺乏完整 oracle。现有方法混合文本投票、排序和基于执行的共识,但各组件贡献不明确。本文研究了18种配置,比较了输出模式多数投票、加权投票、MBR-Exec 和语义投票方法。发现执行基于选择器在所有配置中均优于输出模式多数投票,且输入质量对结果影响更大。此外,思考层次对不同选择方法的影响不同,深度思考提升多数投票效果,但基于执行的方法在候选多样性降低时表现稳定或下降。这些结果表明,在无 oracle 情况下,行为证据比聚合规则更重要。

英文摘要:

LLM code-generation pipelines often sample multiple candidates and select one final answer without access to a complete oracle. Existing pipelines mix textual voting, ranking, and execution-based agreement, but the relative contribution of each component remains unclear. We study 18 configurations across different models, thinking levels, and benchmarks, comparing output-pattern majority voting, weighted voting, MBR-Exec, and SemanticVote - a method that clusters candidates by execution fingerprints on LLM-generated inputs. Three findings emerge. (1) The best execution-based selector exceeds output-pattern majority voting by 19-52 percentage points on every configuration, with every execution-based selector exceeding it by at least 18 points. (2) Once candidates are executed on diverse inputs, aggregation rule has limited effect: SemanticVote, weighted voting, and MBR-Exec are statistically indistinguishable across all 18 configurations. The largest factor is input quality: sketch-based input generation consistently outperforms direct LLM generation by 0.6-2.1 pp and random fuzzing by up to 11.3 pp. (3) Thinking level interacts differently with selection families: deeper thinking improves majority voting by 12 pp but execution-based methods stay flat or degrade as candidate diversity falls. These results frame inference-time code selection as a signal-quality problem rather than an aggregation-rule problem: when oracles are unavailable, the behavioral evidence matters more than the aggregation rule.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑