通过 ChatGPT 进行自协作代码生成
Self-collaboration Code Generation via ChatGPT
AI总结：
本文提出以 ChatGPT 为例的 LLM 自协作代码生成框架，通过分析师、程序员和测试员角色组成虚拟团队，显著提升 Pass@1 并支持复杂仓库级任务。
摘要：
尽管大型语言模型（LLM）已经展现出卓越的代码生成能力，但它们在处理复杂任务时仍然存在困难。在现实软件开发中，人类通常通过协作式团队合作来解决复杂任务，这一策略能够显著控制开发复杂性并提升软件质量。受此启发，我们提出了一种采用 LLM 的代码生成自协作框架，并以 ChatGPT 为例。具体而言，通过角色指令：1）多个 LLM 智能体充当不同的“专家”，每个专家负责复杂任务中的一个特定子任务；2）规定协作与交互方式，使不同角色组成一个虚拟团队来促进彼此的工作，最终该虚拟团队无需人工干预即可协作完成代码生成任务。为了有效组织和管理这一虚拟团队，我们将软件开发方法论纳入该框架。因此，我们组建了一个由三个 LLM 角色（即分析师、程序员和测试员）构成的基础团队，分别负责软件开发的分析、编码和测试阶段。我们在多个代码生成基准上进行了全面实验。实验结果表明，与基础 LLM 智能体相比，自协作代码生成在 Pass@1 上相对提升了 29.9%–47.1%。此外，我们展示了自协作有可能使 LLM 高效处理单一 LLM 智能体不易解决的复杂仓库级任务。
英文摘要：
Although Large Language Models (LLMs) have demonstrated remarkable code-generation ability, they still struggle with complex tasks. In real-world software development, humans usually tackle complex tasks through collaborative teamwork, a strategy that significantly controls development complexity and enhances software quality. Inspired by this, we present a self-collaboration framework for code generation employing LLMs, exemplified by ChatGPT. Specifically, through role instructions, 1) Multiple LLM agents act as distinct `experts', each responsible for a specific subtask within a complex task; 2) Specify the way to collaborate and interact, so that different roles form a virtual team to facilitate each other's work, ultimately the virtual team addresses code generation tasks collaboratively without the need for human intervention. To effectively organize and manage this virtual team, we incorporate software-development methodology into the framework. Thus, we assemble an elementary team consisting of three LLM roles (i.e., analyst, coder, and tester) responsible for software development's analysis, coding, and testing stages. We conduct comprehensive experiments on various code-generation benchmarks. Experimental results indicate that self-collaboration code generation relatively improves 29.9%-47.1% Pass@1 compared to the base LLM agent. Moreover, we showcase that self-collaboration could potentially enable LLMs to efficiently handle complex repository-level tasks that are not readily solved by the single LLM agent.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
