RefineCoder：通过自适应批评精炼迭代改进大型语言模型以用于代码生成
RefineCoder: Iterative Improving of Large Language Models via Adaptive Critique Refinement for Code Generation
AI总结：
本文提出自适应批评精炼方法ACR，通过自生成代码和外部批评迭代改进代码LLM，并据此构建RefineCoder系列，在多个基准上以更少数据取得相当或更优性能。
摘要：
随着大型语言模型（LLMs）的兴起，代码生成受到越来越多的关注。许多研究通过合成与代码相关的指令数据并应用监督微调，开发了强大的代码LLMs。然而，这些方法受限于教师模型蒸馏，并忽略了通过自生成代码进行迭代精炼的潜力。本文提出了自适应批评精炼（Adaptive Critique Refinement，ACR），使模型能够通过自生成代码和外部批评来精炼自身，而不是直接模仿教师模型的代码响应。具体而言，ACR包括一个以LLM作为评判者（LLM-as-a-Judge）的复合评分系统，用于评估代码响应的质量；以及一个以LLM作为批评者（LLM-as-a-Critic）的选择性批评策略，用于批评自生成的低质量代码响应。我们通过迭代应用ACR开发了RefineCoder系列，在多个代码生成基准上实现了持续的性能提升。与相同规模的基线相比，我们提出的RefineCoder系列能够使用更少的数据达到相当甚至更优的性能。
英文摘要：
Code generation has attracted increasing attention with the rise of Large Language Models (LLMs). Many studies have developed powerful code LLMs by synthesizing code-related instruction data and applying supervised fine-tuning. However, these methods are limited by teacher model distillation and ignore the potential of iterative refinement by self-generated code. In this paper, we propose Adaptive Critique Refinement (ACR), which enables the model to refine itself by self-generated code and external critique, rather than directly imitating the code responses of the teacher model. Concretely, ACR includes a composite scoring system with LLM-as-a-Judge to evaluate the quality of code responses and a selective critique strategy with LLM-as-a-Critic to critique self-generated low-quality code responses. We develop the RefineCoder series by iteratively applying ACR, achieving continuous performance improvement on multiple code generation benchmarks. Compared to the baselines of the same size, our proposed RefineCoder series can achieve comparable or even superior performance using less data.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
