通过自反思方法生成代码的等价表示
Generating Equivalent Representations of Code By A Self-Reflection Approach
AI总结:
本文提出一种自反思方法,让两个大语言模型相互协作生成代码的等价表示,并在开放和受限两种设置下验证其有效性,以支持多种软件工程任务。
摘要:
代码的等价表示(Equivalent Representations, ERs)是指与代码本身保持相同语义的文本表示,例如自然语言注释和伪代码。ERs 在软件开发和维护中发挥着关键作用。然而,如何自动生成代码的 ERs 仍然是一个尚未解决的挑战。在本文中,我们提出了一种自反思方法来生成代码的 ERs。该方法使两个大语言模型(Large Language Models, LLMs)相互协作,通过反思过程生成 ER。根据是否对 ER 施加约束,我们的方法可以在开放和受限两种设置下生成 ERs。我们进行了一项实证研究,在两种设置下生成 ERs,并获得了八项发现。(1)在开放设置下生成 ERs。在开放设置中,我们允许 LLMs 在没有任何约束的情况下表示代码,分析生成的 ERs 并揭示五项关键发现。这些发现揭示了 LLMs 如何理解代码中的语法结构、API 和数值计算。(2)在受限设置下生成 ERs。在受限设置中,我们对 ERs 施加约束,例如自然语言注释、伪代码和流程图。这使得我们的方法能够处理一系列软件工程任务。基于我们的实验,我们有三项发现表明,我们的方法能够有效地生成符合特定约束的 ERs,从而支持各种软件工程任务。(3)未来方向。我们还讨论了潜在的未来研究方向,例如为代码生成推导中间语言、探索对 LLM 友好的需求描述,以及进一步支持软件工程任务。我们相信,本文将在研究社区中引发讨论,并激发许多后续研究。
英文摘要:
Equivalent Representations (ERs) of code are textual representations that preserve the same semantics as the code itself, e.g., natural language comments and pseudocode. ERs play a critical role in software development and maintenance. However, how to automatically generate ERs of code remains an open challenge. In this paper, we propose a self-reflection approach to generating ERs of code. It enables two Large Language Models (LLMs) to work mutually and produce an ER through a reflection process. Depending on whether constraints on ERs are applied, our approach generates ERs in both open and constrained settings. We conduct a empirical study to generate ERs in two settings and obtain eight findings. (1) Generating ERs in the open setting. In the open setting, we allow LLMs to represent code without any constraints, analyzing the resulting ERs and uncovering five key findings. These findings shed light on how LLMs comprehend syntactic structures, APIs, and numerical computations in code. (2) Generating ERs in the constrained setting. In the constrained setting, we impose constraints on ERs, such as natural language comments, pseudocode, and flowcharts. This allows our approach to address a range of software engineering tasks. Based on our experiments, we have three findings demonstrating that our approach can effectively generate ERs that adhere to specific constraints, thus supporting various software engineering tasks. (3) Future directions. We also discuss potential future research directions, such as deriving intermediate languages for code generation, exploring LLM-friendly requirement descriptions, and further supporting software engineering tasks. We believe that this paper will spark discussions in research communities and inspire many follow-up studies.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。