RedStar：扩展长思维链数据能否解锁更优的慢推理系统？
RedStar: Does Scaling Long-CoT Data Unlock Better Slow-Reasoning Systems?
AI总结：
本文提出RedStar，通过扩展到1000k长思维链数据并引入RL训练，显著提升慢推理性能，在MATH-Hard和AIME等基准上大幅超越基线，验证了长思维链的样本效率和扩展潜力。
摘要：
扩展能否改变推理能力？在这项工作中，我们探索了将长思维链（Long-CoT）数据扩展到1000k样本的未开发潜力，开创性地开发了一个慢思考模型RedStar。通过对各种LLM和不同规模的广泛实验，我们揭示了长思维链训练的专业化和规模化的关键要素。令人惊讶的是，即使是较小的模型在有限的数据下也显示出显著的性能提升，揭示了长思维链的样本效率以及样本难度在学习过程中的关键作用。我们的发现表明，仅需几千个示例即可有效触发长思维链推理，而更大的模型则实现了无与伦比的改进。我们还引入了强化学习（RL）规模的训练，作为推进慢推理系统的一个有前景的方向。RedStar在多个领域表现出色：在MATH-Hard基准上，RedStar-code-math将性能从66.2%提升到81.6%，在美国数学奥林匹克（AIME）中，它仅使用21k混合代码数学数据集就解决了46.7%的问题。在多模态任务如GeoQA和MathVista-GEO中，RedStar-Geo以最少的长思维链数据取得了有竞争力的结果，优于其他慢推理系统如QvQ-Preview。与QwQ相比，RedStar在推理和泛化能力之间取得了完美的平衡。我们的工作强调，通过精心调优，扩展长思维链可以解锁非凡的推理能力——即使数据集有限，也为各种挑战中的慢推理模型树立了新的标准。我们的数据和模型已在https://huggingface.co/RedStar-Reasoning发布。
英文摘要：
Can scaling transform reasoning? In this work, we explore the untapped potential of scaling Long Chain-of-Thought (Long-CoT) data to 1000k samples, pioneering the development of a slow-thinking model, RedStar. Through extensive experiments with various LLMs and different sizes, we uncover the ingredients for specialization and scale for Long-CoT training. Surprisingly, even smaller models show significant performance gains with limited data, revealing the sample efficiency of Long-CoT and the critical role of sample difficulty in the learning process. Our findings demonstrate that Long-CoT reasoning can be effectively triggered with just a few thousand examples, while larger models achieve unparalleled improvements. We also introduce reinforcement learning (RL)-scale training as a promising direction for advancing slow-thinking systems. RedStar shines across domains: on the MATH-Hard benchmark, RedStar-code-math boosts performance from 66.2\% to 81.6\%, and on the USA Math Olympiad (AIME), it solves 46.7\% of problems using only 21k mixed-code-math datasets. In multimodal tasks like GeoQA and MathVista-GEO, RedStar-Geo achieves competitive results with minimal Long-CoT data, outperforming other slow-thinking systems like QvQ-Preview. Compared to QwQ, RedStar strikes the perfect balance between reasoning and generalizability. Our work highlights that, with careful tuning, scaling Long-CoT can unlock extraordinary reasoning capabilities-even with limited dataset and set a new standard for slow-thinking models across diverse challenges. Our data and models are released at https://huggingface.co/RedStar-Reasoning.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
