arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2603.07779cs.CLcs.GLcs.LG
收藏

数据难度的扩展:通过在新鲜且具有挑战性的问题上进行强化学习来改进编码模型

Scaling Data Difficulty: Improving Coding Models via Reinforcement Learning on Fresh and Challenging Problems

Microsoft Research(微软研究院);University of Cambridge(剑桥大学)

Zongqian Li, Tengchao Lv, Shaohan Huang, Yixuan Su, Qinzheng Sun, Qiufeng Yin, Ying Xin, Scarlett Li, Lei Cui, Nigel Collier, Furu Wei

AI总结:

通过强化学习在新鲜且具有挑战性的问题上改进编码模型,MicroCoder数据集通过系统化数据处理和难度扩展实现了显著的性能提升。

摘要:

训练下一代代码生成模型需要高质量的数据集,但现有数据集面临难度不平衡、格式不一致和数据质量的问题。我们通过系统化的数据处理和难度扩展来解决这些挑战。我们引入了一个四阶段的数据处理框架,包括收集、处理、过滤和验证,其中包含基于LLM的自动难度过滤框架,该框架利用多维难度度量标准在五个加权维度上保留具有挑战性的问题并去除简单的问题。所得到的MicroCoder数据集包含成千上万经过精心挑选的来自不同平台的真实编程竞赛问题,强调时效性和难度。在严格未见过的LiveCodeBench上的评估显示,MicroCoder在300训练步骤内相比现有基准数据集获得了3倍大的性能提升,且在GRPO及其变种训练算法下保持一致优势。MicroCoder数据集在不同模型大小上对中等和困难问题实现了明显改进,在模型能力最受限时达到整体性能的17.2%相对提升。这些结果验证了难度意识的数据整理能够提高模型在具有挑战性任务上的性能,为代码生成的数据集创建提供了多个见解。

英文摘要:

Training next-generation code generation models requires high-quality datasets, yet existing datasets face difficulty imbalance, format inconsistency, and data quality problems. We address these challenges through systematic data processing and difficulty scaling. We introduce a four-stage Data Processing Framework encompassing collection, processing, filtering, and verification, incorporating Automatic Difficulty Filtering via an LLM-based predict-calibrate-select framework that leverages multi-dimensional difficulty metrics across five weighted dimensions to retain challenging problems while removing simplistic ones. The resulting MicroCoder dataset comprises tens of thousands of curated real competitive programming problems from diverse platforms, emphasizing recency and difficulty. Evaluations on strictly unseen LiveCodeBench demonstrate that MicroCoder achieves 3x larger performance gains within 300 training steps compared to widely-used baseline datasets of comparable size, with consistent advantages under both GRPO and its variant training algorithms. The MicroCoder dataset delivers obvious improvements on medium and hard problems across different model sizes, achieving up to 17.2% relative gains in overall performance where model capabilities are most stretched. These results validate that difficulty-aware data curation improves model performance on challenging tasks, providing multiple insights for dataset creation in code generation.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑