OpenCodeReasoning：推进竞技编程的数据蒸馏
OpenCodeReasoning: Advancing Data Distillation for Competitive Coding
AI总结：
本文提出OpenCodeReasoning，通过构建优质监督微调数据集，仅用SFT在竞技编程基准上超越强化学习模型，并分析数据源、过滤及多样性影响，开源数据集与模型。
摘要：
自从基于推理的大型语言模型问世以来，许多研究者通过将推理能力蒸馏到学生模型中取得了巨大成功。这些技术显著缩小了推理模型与标准大语言模型在编码任务上的差距。尽管如此，推理模型蒸馏方面的许多进展仍被锁定在专有数据集之中，或者缺乏关于数据整理、过滤和后续训练的细节。为了解决这一问题，我们构建了一个优质的监督微调（SFT）数据集，并利用该数据集在多种规模的模型中实现了最先进的编码能力结果。我们的蒸馏模型仅使用SFT就在LiveCodeBench上达到61.8%的准确率，在CodeContests上达到24.6%，超越了使用强化学习训练的替代方案。随后，我们对构建数据集所用的数据源、代码执行过滤的影响以及指令/解决方案多样性的重要性进行了分析。我们观察到执行过滤对基准准确率产生了负面影响，这促使我们将指令多样性置于解决方案正确性之上。最后，我们还分析了这些模型使用的令牌效率和推理模式。我们将向社区开源这些数据集和蒸馏模型。
英文摘要：
Since the advent of reasoning-based large language models, many have found great success from distilling reasoning capabilities into student models. Such techniques have significantly bridged the gap between reasoning and standard LLMs on coding tasks. Despite this, much of the progress on distilling reasoning models remains locked behind proprietary datasets or lacks details on data curation, filtering and subsequent training. To address this, we construct a superior supervised fine-tuning (SFT) dataset that we use to achieve state-of-the-art coding capability results in models of various sizes. Our distilled models use only SFT to achieve 61.8% on LiveCodeBench and 24.6% on CodeContests, surpassing alternatives trained with reinforcement learning. We then perform analysis on the data sources used to construct our dataset, the impact of code execution filtering, and the importance of instruction/solution diversity. We observe that execution filtering negatively affected benchmark accuracy, leading us to prioritize instruction diversity over solution correctness. Finally, we also analyze the token efficiency and reasoning patterns utilized by these models. We will open-source these datasets and distilled models to the community.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
