通过学习多样的思维链模式扩展基础模型的推理潜力
Expanding Reasoning Potential in Foundation Model by Learning Diverse Chains of Thought Patterns
AI总结：
本文提出通过学习富含高价值推理模式的多样化数据，扩展基础模型的推理潜力，显著提升模型在数学推理任务上的性能。
摘要：
近期大型推理模型在解决具有挑战性的数学推理问题上的进步主要由强化学习（RL）推动。在中期训练过程中整合长链思维（CoT）数据也被证明能显著提高推理深度。然而，当前方法往往随意使用CoT数据，留下了关键问题：哪些数据类型最能增强模型的推理能力。在本文中，我们首次定义了基础模型的推理潜力为正确回答问题所需独立尝试次数的倒数，这与最终模型性能有很强的相关性。我们随后提出利用富含高价值推理模式的多样化数据来扩展推理潜力。具体而言，我们从CoT序列中抽象出原子推理模式，这些模式具有共同性和归纳能力，并用它们构建一个富含有价值推理模式的核心参考集。此外，我们提出了一种双粒度算法，涉及推理模式的链条和令牌熵，高效地从数据池中选择与核心集一致的高价值CoT数据（CoTP），从而训练模型有效掌握推理。仅10B-token的CoTP数据使85A6B混合专家（MoE）模型在具有挑战性的AIME 2024和2025上提高了9.58%，并提高了下游RL性能的上限7.81%。
英文摘要：
Recent progress in large reasoning models for challenging mathematical reasoning has been driven by reinforcement learning (RL). Incorporating long chain-of-thought (CoT) data during mid-training has also been shown to substantially improve reasoning depth. However, current approaches often utilize CoT data indiscriminately, leaving open the critical question of which data types most effectively enhance model reasoning capabilities. In this paper, we define the foundation model's reasoning potential for the first time as the inverse of the number of independent attempts required to correctly answer the question, which is strongly correlated with the final model performance. We then propose utilizing diverse data enriched with high-value reasoning patterns to expand the reasoning potential. Specifically, we abstract atomic reasoning patterns from CoT sequences, characterized by commonality and inductive capabilities, and use them to construct a core reference set enriched with valuable reasoning patterns. Furthermore, we propose a dual-granularity algorithm involving chains of reasoning patterns and token entropy, efficiently selecting high-value CoT data (CoTP) from the data pool that aligns with the core set, thereby training models to master reasoning effectively. Only 10B-token CoTP data enables the 85A6B Mixture-of-Experts (MoE) model to improve by 9.58% on the challenging AIME 2024 and 2025, and to raise the upper bound of downstream RL performance by 7.81%.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
