大型语言模型(LLMs)的进步极大地推动了代码生成领域的发展。以往的工作将强化学习(RL)与编译器反馈相结合,用于探索LLMs的输出空间,以提升代码生成质量。然而,LLMs针对复杂人类需求生成的长代码使得RL探索面临挑战。此外,由于单元测试可能无法覆盖复杂代码,利用这些未执行的代码片段来优化LLMs是低效的。为应对这些挑战,我们引入了StepCoder,一种新颖的用于代码生成的RL框架,包含两个主要组件:CCCS通过将长序列代码生成任务分解为代码补全子任务的课程(Curriculum of Code Completion Subtasks)来解决探索挑战,而FGO则通过掩蔽未执行的代码段来提供细粒度优化(Fine-Grained Optimization),仅优化模型。此外,我们还构建了APPS+数据集用于RL训练,该数据集经过人工验证以确保单元测试的正确性。实验结果表明,我们的方法提升了探索输出空间的能力,并在相应基准测试中超越了最先进的方法。我们的数据集APPS+和StepCoder已在线提供。
英文摘要:
The advancement of large language models (LLMs) has significantly propelled the field of code generation. Previous work integrated reinforcement learning (RL) with compiler feedback for exploring the output space of LLMs to enhance code generation quality. However, the lengthy code generated by LLMs in response to complex human requirements makes RL exploration a challenge. Also, since the unit tests may not cover the complicated code, optimizing LLMs by using these unexecuted code snippets is ineffective. To tackle these challenges, we introduce StepCoder, a novel RL framework for code generation, consisting of two main components: CCCS addresses the exploration challenge by breaking the long sequences code generation task into a Curriculum of Code Completion Subtasks, while FGO only optimizes the model by masking the unexecuted code segments to provide Fine-Grained Optimization. In addition, we furthermore construct the APPS+ dataset for RL training, which is manually verified to ensure the correctness of unit tests. Experimental results show that our method improves the ability to explore the output space and outperforms state-of-the-art approaches in corresponding benchmarks. Our dataset APPS+ and StepCoder are available online.
Compilable Neural Code Generation with Compiler Feedback
Wuhan University(武汉大学);Huawei Noah’s Ark Lab(华为诺亚方舟实验室);Huazhong University of Science and Technology(华中科技大学);Huawei Technologies Co., Ltd.(华为技术有限公司);Yunnan University(云南大学)
CodeEvo: Interaction-Driven Synthesis of Code-centric Data through Hybrid and Iterative Feedback
Shanghai AI Laboratory(上海人工智能实验室);The University of Hong Kong(香港大学);New York University(纽约大学);Carnegie Mellon University(卡内基梅隆大学);Shanghai Innovation Institute(上海创新研究院)
Fudan University(复旦大学);Peking University(北京大学);Nanyang Technological University(南洋理工大学);Huazhong University of Science and Technology(华中科技大学);Harbin Institute of Technology(哈尔滨工业大学);Meituan Inc(美团公司)
LEGO-Compiler: Enhancing Neural Compilation Through Translation Composability
SKLP, Institute of Computing Technology, CAS University of Chinese Academy of Sciences, Beijing, China(SKLP,计算技术研究所,中国科学院大学,北京,中国);University of Leeds, UK(利兹大学,英国);Zhongguancun Laboratory, Beijing, China(中关村实验室,北京,中国)
GCoder: Improving Large Language Model for Generalized Graph Problem Solving
Hong Kong University of Science and Technology (GuangZhou)(香港科技大学(广州));State Key Laboratory for Novel Software Technology, Nanjing University(南京大学计算机软件新技术国家重点实验室)
FeedbackEval: A Benchmark for Evaluating Large Language Models in Feedback-Driven Code Repair Tasks
School of Software Engineering Sun Yat-sen University(中山大学软件学院);The Hong Kong University of Science and Technology(香港科技大学);School of Computer Science and Engineering Nanyang Technological University(南洋理工大学计算机科学与工程学院);School of Computer Science Fudan University(复旦大学计算机科学学院)
Interleaved Learning and Exploration: A Self-Adaptive Fuzz Testing Framework for MLIR
Institute of Software Chinese Academy of Sciences Beijing, China;College of Intelligence;Computing Tianjin University Tianjin, China;Shanghai Key Laboratory of Trustworthy Computing East China Normal University Shanghai, China
CHISEL-ing Back Source Code with AI-enabled Iterative Recovery
A*STAR Institute of Advanced Intelligence and Computing(A*STAR先进智能与计算研究所);Coimbatore Institution of Technology(科因巴托尔理工学院);National University of Singapore(新加坡国立大学)
Beyond Code Pairs: Dialogue-Based Data Generation for LLM Code Translation
Argonne National Laboratory(阿贡国家实验室);University of Minnesota(明尼苏达大学);Iowa State University(爱荷华州立大学);Lawrence Livermore National Laboratory(劳伦斯利弗莫尔国家实验室)
SPARC: Scenario Planning and Reasoning for Automated C Unit Test Generation
Bangladesh University of Engineering and Technology(孟加拉工程科技大学);National Taiwan University(台湾国立大学);University of Illinois at Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校)