arXiv 2508.052422025-08-07cs.CL

CodeBoost：通过强化学习从代码片段中提取知识提升代码大语言模型

CodeBoost: Boosting Code LLMs by Squeezing Knowledge from Code Snippets with RL

Nanyang Technological University（南洋理工大学）；University of Electronic Science and Technology of China（电子科技大学）；Beihang University（北京航空航天大学）；Northeastern University, China（中国东北大学）

Sijie Wang, Quanjiang Guo, Kai Zhao, Yawei Zhang, Xin Li, Xiang Li, Siqi Li, Rui She, Shangshu Yu, Wee Peng Tay

备注：: Technical report. Project page: https://github.com/sijieaaa/CodeBoost

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

CodeBoost提出一种仅从代码片段后训练代码大语言模型的框架，通过最大团筛选、双向预测、错误感知、异构增强与奖励，在多个基准上持续提升性能。

摘要：

代码大语言模型（LLM）已成为构建高效自动化编码流程不可或缺的工具。现有模型通常使用通用大语言模型，基于“人工指令-最终答案”对进行强化学习（RL）后训练，其中指令通常来自人工标注。然而，收集高质量的编码指令既耗费人力又难以规模化。另一方面，代码片段可从多种来源大量获取。这种不平衡构成了基于指令的后训练的主要瓶颈。我们提出CodeBoost，一种后训练框架，仅从代码片段增强代码大语言模型，不依赖人工标注指令。CodeBoost引入以下关键组件：（1）最大团筛选，从代码中选择具有代表性和多样性的训练语料；（2）双向预测，使模型能够从正向和反向预测目标中学习；（3）错误感知预测，结合正确和错误输出的学习信号；（4）异构增强，多样化训练分布以丰富代码语义；（5）异构奖励，通过多种奖励类型（包括格式正确性和来自成功与失败的执行反馈）指导模型学习。在多个代码大语言模型和基准上的大量实验验证了CodeBoost持续提升性能，证明了其作为可扩展且有效的训练流程的有效性。

英文摘要：

Code large language models (LLMs) have become indispensable tools for building efficient and automated coding pipelines. Existing models are typically post-trained using reinforcement learning (RL) from general-purpose LLMs using "human instruction-final answer" pairs, where the instructions are usually from manual annotations. However, collecting high-quality coding instructions is both labor-intensive and difficult to scale. On the other hand, code snippets are abundantly available from various sources. This imbalance presents a major bottleneck in instruction-based post-training. We propose CodeBoost, a post-training framework that enhances code LLMs purely from code snippets, without relying on human-annotated instructions. CodeBoost introduces the following key components: (1) maximum-clique curation, which selects a representative and diverse training corpus from code; (2) bi-directional prediction, which enables the model to learn from both forward and backward prediction objectives; (3) error-aware prediction, which incorporates learning signals from both correct and incorrect outputs; (4) heterogeneous augmentation, which diversifies the training distribution to enrich code semantics; and (5) heterogeneous rewarding, which guides model learning through multiple reward types including format correctness and execution feedback from both successes and failures. Extensive experiments across several code LLMs and benchmarks verify that CodeBoost consistently improves performance, demonstrating its effectiveness as a scalable and effective training pipeline.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。