arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2512.07404cs.SEcs.AIcs.LG
收藏

关于LLMs内部对代码正确性的表示

On LLMs' Internal Representation of Code Correctness

New York University Abu Dhabi(纽约大学阿布扎比分校)

Francisco Ribeiro, Claudio Spiess, Prem Devanbu, Sarah Nadi

AI总结:

本文研究了LLMs内部对代码正确性的表示,通过对比隐藏状态识别正确性表示,并展示其在提升代码生成质量与可靠性方面的贡献。

摘要:

尽管大型语言模型(LLMs)在代码生成方面表现出色,但它们经常生成错误的代码。其中一个原因是模型输出概率往往与正确性不密切相关,仅反映生成过程的最终输出。受发现LLMs内部编码诸如真实性等概念的启发,本文探讨LLMs是否同样表示代码正确性。具体而言,我们通过对比相同编程任务中正确和错误代码的隐藏状态,识别出LLMs内部的正确性表示。通过在四个LLMs上进行实验,我们展示了利用提取的正确性表示可以优于标准对数似然排名以及口头化的模型置信度。此外,我们探讨了如何利用这种内部正确性信号来选择高质量的代码样本,而无需测试执行。最终,这项工作展示了如何利用内部表示来增强代码生成系统,使LLMs更加可靠,从而提高自动生成功能的置信度。

英文摘要:

Despite the effectiveness of large language models (LLMs) for code generation, they often output incorrect code. One reason is that model output probabilities are often not well-correlated with correctness, and reflect only the final output of the generation process. Inspired by findings that LLMs internally encode concepts like truthfulness, this paper explores if LLMs similarly represent code correctness. Specifically, we identify a correctness representation inside LLMs by contrasting the hidden states between pairs of correct and incorrect code for the same programming tasks. By experimenting on four LLMs, we show that exploiting this extracted correctness representation outperforms standard log-likelihood ranking, as well as verbalized model confidence. Furthermore, we explore how this internal correctness signal can be used to select higher-quality code samples, without requiring test execution. Ultimately, this work demonstrates how leveraging internal representations can enhance code generation systems and make LLMs more reliable, thus improving confidence in automatically generated code.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑