arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2501.02138cs.PLcs.AIcs.SE
收藏

使用 Pythoness 实现有效的 LLM 驱动代码生成

Effective LLM-Driven Code Generation with Pythoness

University of Massachusetts Amherst(马萨诸塞大学阿默斯特分校);Williams College(威廉姆斯学院);Amazon(亚马逊公司)

Kyla H. Levin, Kyle Gwilt, Emery D. Berger, Stephen N. Freund

AI总结:

本文提出嵌入式 DSL Pythoness,让开发者通过单元测试、属性测试等行为规范引导 LLM 生成并持续验证代码,从而提升代码质量并降低 LLM 代码生成风险。

摘要:

大型语言模型(LLM)的出现为编程工具的新时代铺平了道路,这些工具既具备显著能力,也伴随着风险,因为生成的代码缺乏正确性和可靠性保证。当前使用 LLM 的开发者面临着优化、集成和维护 AI 生成代码的艰巨任务。我们提出一种嵌入式领域特定语言(DSL)Pythoness,以应对这些挑战。在 Pythoness 中,开发者在更高的抽象层级上使用 LLM 编程。使用 Pythoness 的开发者不会直接与生成的代码交互,而是在编写函数、类或整个程序时,在行为规范的层面上进行操作。这些规范可以采用单元测试和基于属性的测试形式,并且可以形式化表达,也可以用自然语言表达。在这些规范的引导下,Pythoness 生成既能够通过测试、又可在执行期间持续检查的代码。我们认为,Pythoness 方法使开发者能够充分发挥 LLM 在代码生成方面的潜力,同时显著降低其固有风险。我们描述了当前的 Pythoness 原型实现,并证明它能够成功结合测试与代码生成,产生比仅使用规范更高质量的代码。

英文摘要:

The advent of large language models (LLMs) has paved the way for a new era of programming tools with both significant capabilities and risks, as the generated code lacks guarantees of correctness and reliability. Developers using LLMs currently face the difficult task of optimizing, integrating, and maintaining code generated by AI. We propose an embedded domain-specific language (DSL), Pythoness, to address those challenges. In Pythoness, developers program with LLMs at a higher level of abstraction. Rather than interacting directly with generated code, developers using Pythoness operate at the level of behavioral specifications when writing functions, classes, or an entire program. These specifications can take the form of unit tests and property-based tests, which may be expressed formally or in natural language. Guided by these specifications, Pythoness generates code that both passes the tests and can be continuously checked during execution. We posit that the Pythoness approach lets developers harness the full potential of LLMs for code generation while substantially mitigating their inherent risks. We describe our current prototype implementation of Pythoness and demonstrate that it can successfully leverage a combination of tests and code generation to yield higher quality code than specifications alone.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑