CodeChain：通过代表性子模块的自我修正链实现模块化代码生成
CodeChain: Towards Modular Code Generation Through Chain of Self-revisions with Representative Sub-modules
AI总结：
CodeChain通过自我修正链和代表性子模块聚类重用，显著提升LLM在复杂编程任务中的模块化代码生成正确性，在APPS和CodeContests上分别提升35%和76%的pass@1。
摘要：
大型语言模型（LLMs）在解决诸如HumanEval或MBPP基准测试中的简单编程任务时已经相当熟练。然而，解决更复杂和更具竞争性的编程任务对这些模型来说仍然相当具有挑战性——这可能是由于它们倾向于将解决方案生成为整体代码块，而不是将其分解为逻辑子任务和子模块。另一方面，经验丰富的程序员本能地编写带有抽象性的模块化代码来解决复杂任务，通常重用先前开发的模块。为了解决这一差距，我们提出了CodeChain，一种新颖的推理框架，通过一系列自我修正来引发模块化代码生成，每次修正由先前迭代中生成的一些代表性子模块引导。具体而言，CodeChain首先通过思维链提示指导LLM生成模块化代码。然后，它通过迭代两个步骤应用自我修正链：1）提取并聚类生成的子模块，选择聚类代表作为更通用和可重用的实现；2）用这些选定的模块实现增强原始思维链提示，并指示LLM重新生成新的模块化解决方案。我们发现，通过自然鼓励LLM重用先前开发和验证的子模块，CodeChain可以显著提高生成解决方案的模块性和正确性，在APPS上实现相对pass@1改进35%，在CodeContests上实现76%。它被证明对OpenAI LLM以及像WizardCoder这样的开源LLM都有效。我们还进行了全面的消融研究，采用不同的提示方法、聚类数量、模型大小、程序质量等，以提供支撑CodeChain成功的有用见解。
英文摘要：
Large Language Models (LLMs) have already become quite proficient at solving simpler programming tasks like those in HumanEval or MBPP benchmarks. However, solving more complex and competitive programming tasks is still quite challenging for these models - possibly due to their tendency to generate solutions as monolithic code blocks instead of decomposing them into logical sub-tasks and sub-modules. On the other hand, experienced programmers instinctively write modularized code with abstraction for solving complex tasks, often reusing previously developed modules. To address this gap, we propose CodeChain, a novel framework for inference that elicits modularized code generation through a chain of self-revisions, each being guided by some representative sub-modules generated in previous iterations. Concretely, CodeChain first instructs the LLM to generate modularized codes through chain-of-thought prompting. Then it applies a chain of self-revisions by iterating the two steps: 1) extracting and clustering the generated sub-modules and selecting the cluster representatives as the more generic and re-usable implementations, and 2) augmenting the original chain-of-thought prompt with these selected module-implementations and instructing the LLM to re-generate new modularized solutions. We find that by naturally encouraging the LLM to reuse the previously developed and verified sub-modules, CodeChain can significantly boost both modularity as well as correctness of the generated solutions, achieving relative pass@1 improvements of 35% on APPS and 76% on CodeContests. It is shown to be effective on both OpenAI LLMs as well as open-sourced LLMs like WizardCoder. We also conduct comprehensive ablation studies with different methods of prompting, number of clusters, model sizes, program qualities, etc., to provide useful insights that underpin CodeChain's success.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
