arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2303.06689cs.SE
收藏

使用大型语言模型进行自规划代码生成

Self-planning Code Generation with Large Language Models

Key Laboratory of High Confidence Software Technologies (Peking University), Ministry of Education(教育部高可信软件技术重点实验室);School of Computer Science, Peking University(北京大学计算机学院)

Xue Jiang, Yihong Dong, Lecheng Wang, Zheng Fang, Qiwei Shang, Ge Li, Zhi Jin, Wenpin Jiao

AI总结:

本文提出 LLM 自规划代码生成方法,通过先制定格式化规划、再逐步实现代码,显著提升复杂意图下的 Pass@1 表现与代码质量。

摘要:

尽管大型语言模型(LLM)在代码生成方面展现出令人瞩目的能力,但它们仍难以处理人类提供的复杂意图。人们普遍认为,人类通常会运用规划来分解复杂问题,并在实现之前安排解决步骤。为此,我们将规划引入代码生成,以帮助模型理解复杂意图并降低解决问题的难度。本文提出了一种使用大型语言模型的自规划代码生成方法,该方法由两个阶段组成,即规划阶段和实现阶段。具体而言,在规划阶段,LLM根据意图概述简洁且格式化的规划步骤。随后,在实现阶段,模型在先前规划步骤的指导下逐步生成代码。我们在多种编程语言的各类代码生成基准上进行了大量实验。实验结果表明,与直接代码生成相比,自规划代码生成在 Pass@1 上实现了高达 25.4% 的相对提升,而与 Chain-of-Thought(思维链)代码生成相比则实现了高达 11.9% 的相对提升。此外,经人类评估,我们的自规划方法还在正确性、可读性和健壮性方面提升了生成代码的质量。

英文摘要:

Although large language models (LLMs) have demonstrated impressive ability in code generation, they are still struggling to address the complicated intent provided by humans. It is widely acknowledged that humans typically employ planning to decompose complex problems and schedule solution steps prior to implementation. To this end, we introduce planning into code generation to help the model understand complex intent and reduce the difficulty of problem-solving. This paper proposes a self-planning code generation approach with large language models, which consists of two phases, namely planning phase and implementation phase. Specifically, in the planning phase, LLM outlines concise and formatted planning steps from the intent. Subsequently, in the implementation phase, the model generates code step by step, guided by the preceding planning steps. We conduct extensive experiments on various code-generation benchmarks across multiple programming languages. Experimental results show that self-planning code generation achieves a relative improvement of up to 25.4% in Pass@1 compared to direct code generation, and up to 11.9% compared to Chain-of-Thought code generation. Moreover, our self-planning approach also enhances the quality of the generated code with respect to correctness, readability, and robustness, as assessed by humans.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑