基于人类偏好评估并对齐代码大语言模型
Evaluating and Aligning CodeLLMs on Human Preference
AI总结：
本文构建贴近真实任务的CodeArena基准，并利用近20B tokens的SynCode-Instruct验证合成指令微调，表明开源代码LLM与专有模型在人类偏好对齐上仍有明显差距。
摘要：
代码大语言模型（codeLLMs）在代码生成方面取得了显著进展。以往大多数代码相关基准由各种编程练习及相应测试用例组成，被用作评估代码LLM性能和能力的通用衡量标准。然而，当前代码LLM专注于合成正确的代码片段，忽视了与人类偏好的对齐，即查询应从实际应用场景中采样，且模型生成的响应应满足人类偏好。为弥合模型生成响应与人类偏好之间的差距，我们提出了一个严格的人工整理基准CodeArena，用以模拟现实编码任务的复杂性和多样性；其中包含从用户查询中精心整理的397个高质量样本，覆盖40个类别和44种编程语言。此外，我们通过扩展来自网站的指令，提出了一个多样化的合成指令语料库SynCode-Instruct（近20B tokens），以验证大规模合成指令微调的有效性；完全基于合成指令数据训练的Qwen2.5-SynCoder能够达到开源代码LLM中的顶级性能。结果发现，基于执行的基准与CodeArena之间存在性能差异。我们在CodeArena上对40多个LLM进行的系统实验揭示，开放的SOTA代码LLM（例如Qwen2.5-Coder）与专有LLM（例如OpenAI o1）之间存在显著性能差距，凸显了人类偏好对齐的重要性。
英文摘要：
Code large language models (codeLLMs) have made significant strides in code generation. Most previous code-related benchmarks, which consist of various programming exercises along with the corresponding test cases, are used as a common measure to evaluate the performance and capabilities of code LLMs. However, the current code LLMs focus on synthesizing the correct code snippet, ignoring the alignment with human preferences, where the query should be sampled from the practical application scenarios and the model-generated responses should satisfy the human preference. To bridge the gap between the model-generated response and human preference, we present a rigorous human-curated benchmark CodeArena to emulate the complexity and diversity of real-world coding tasks, where 397 high-quality samples spanning 40 categories and 44 programming languages, carefully curated from user queries. Further, we propose a diverse synthetic instruction corpus SynCode-Instruct (nearly 20B tokens) by scaling instructions from the website to verify the effectiveness of the large-scale synthetic instruction fine-tuning, where Qwen2.5-SynCoder totally trained on synthetic instruction data can achieve top-tier performance of open-source code LLMs. The results find performance differences between execution-based benchmarks and CodeArena. Our systematic experiments of CodeArena on 40+ LLMs reveal a notable performance gap between open SOTA code LLMs (e.g. Qwen2.5-Coder) and proprietary LLMs (e.g., OpenAI o1), underscoring the importance of the human preference alignment.\footnote{\url{https://codearenaeval.github.io/ }}
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
