arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2606.09800cs.SEcs.AIcs.MA
收藏

FASE: 用于代码质量的快速自适应语义熵

FASE: Fast Adaptive Semantic Entropy for Code Quality

University of Waterloo(滑铁卢大学)

Shizhe Lin, Ladan Tahvildari

AI总结:

提出快速自适应语义熵(FASE),通过最小生成树近似功能正确性,在HumanEval和BigCodeBench上相比现有语义熵方法在Spearman相关性和ROCAUC上分别提升25%和19%,且计算开销仅为传统方法的0.3%。

摘要:

多智能体代码生成通过模拟人类软件工程生命周期,为自主软件开发提供了一种有前景的范式。然而,系统可靠性仍然受到LLM幻觉和跨交互智能体错误传播的阻碍。虽然语义熵提供了一种无需真实答案即可量化不确定性的原则性方法,但当前方法通常依赖于成本高昂的LLM驱动的等价性检查。在这项工作中,我们引入了快速自适应语义熵(FASE),这是一种基于结构和语义不相似图的最小生成树来近似功能正确性的新型度量。在HumanEval和BigCodeBench上的评估表明,FASE优于通过LLM蕴含的最先进语义熵,在使用Qwen3-Embedding-8B模型时,与基于真实测试用例的Pass@1相比,Spearman相关性平均提升25%,ROCAUC分数提升19%。此外,通过消除成本高昂的LLM驱动的等价性评估,FASE的计算开销可忽略不计,其运行成本仅为传统语义熵方法的约0.3%。这些结果使FASE成为优化现实世界多智能体工作流中不确定性量化的实用且经济高效的解决方案。

英文摘要:

Multi-agent code generation offers a promising paradigm for autonomous software development by simulating the human software engineering lifecycle. However, system reliability remains hindered by LLM hallucinations and error propagation across interacting agents. While semantic entropy provides a principled way to quantify uncertainty without ground-truth answers, current methods often rely on costly LLM-driven equivalence checks. In this work, we introduce Fast Adaptive Semantic Entropy (FASE), a novel metric that approximates functional correctness based on the minimum spanning tree of structural and semantic dissimilarity graphs. Evaluations on HumanEval and BigCodeBench demonstrate that FASE outperforms state-of-the-art semantic entropy by LLM entailment, achieving a 25% average improvement in Spearman correlation and a 19% increase in ROCAUC score against Pass@1 from ground-truth test cases when using the Qwen3-Embedding-8B model. Furthermore, by eliminating costly LLM-driven equivalence evaluation, FASE incurs negligible computational overhead, requiring only approximately 0.3% of the runtime cost of traditional semantic entropy approaches. These results position FASE as a practical, cost-effective solution for optimizing uncertainty quantification in real-world multi-agent workflows.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑