arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.01187cs.SEcs.AI
收藏

自回归 yet 可修订:在解码修订中实现安全代码生成

Autoregressive, Yet Revisable: In Decoding Revision for Secure Code Generation

Chengran Yang, Zichao Wei, Heminghao Deng, Jinfeng Jiang, Zhensu Sun, Ting Zhang, Tianyi Wu, Ming Wen, David Lo

AI总结:

本文提出“修订流”范式,通过引入特定操作令牌实现代码生成的动态自我纠正,有效减少安全代码生成中的漏洞。

摘要:

基于大型语言模型(LLM)的代码生成通常被建模为严格单调的过程,将令牌线性地附加到一个不可变的前缀上。这种建模方式与编程的认知过程截然不同,后者本质上是与正向生成和即时修订交织在一起的。虽然先前的工作试图通过事后代理或外部静态工具引入修订,但它们要么导致高延迟,要么无法利用模型的内在语义推理能力。在本文中,我们提出了“修订流”这一范式转变,通过利用模型的内在能力,将代码生成从单调流提升为动态、自我纠正的轨迹。我们引入了特定的操作令牌,使模型能够在单次正向传递中无缝回溯并编辑其自身的历史。通过内化修订循环,我们的框架“修订流”允许模型在需要时激活其潜在能力,而无需外部依赖。在安全代码生成的实证结果中,显示“修订流”显著减少了漏洞,同时保持了最小的推理开销。

英文摘要:

Large Language Model (LLM) based code generation is predominantly formulated as a strictly monotonic process, appending tokens linearly to an immutable prefix. This formulation contrasts to the cognitive process of programming, which is inherently interleaved with forward generation and on-the-fly revision. While prior works attempt to introduce revision via post-hoc agents or external static tools, they either suffer from high latency or fail to leverage the model's intrinsic semantic reasoning. In this paper, we propose Stream of Revision, a paradigm shift that elevates code generation from a monotonic stream to a dynamic, self-correcting trajectory by leveraging model's intrinsic capabilities. We introduce specific action tokens that enable the model to seamlessly backtrack and edit its own history within a single forward pass. By internalizing the revision loop, our framework Stream of Revision allows the model to activate its latent capabilities just-in-time without external dependencies. Empirical results on secure code generation show that Stream of Revision significantly reduces vulnerabilities with minimal inference overhead.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑