Post-training Small Language Models (SLMs) for reasoning typically follows an SFT-then-RL pipeline, yet existing work rarely considers what data should be learned at each stage. We argue that data strategy should be aligned with the distinct roles of SFT and RL: SFT is better suited for acquiring not-yet-mastered reasoning skills, while RL is better suited for consolidating skills that the model can already partially access. Based on this principle, we propose a difficulty-aware SFT-then-RL framework that organizes training data into stage-specific sets. For hard samples in the SFT stage, we introduce a Bridge mechanism that transforms raw teacher-generated reasoning traces into more learnable supervision for SLMs. For hard samples that remain unsolved during RL, we apply Critique Fine-Tuning by converting all-zero-reward failures into diagnostic, repair, and new reasoning trace supervision for the next SFT stage. Experiments on two SLMs across five reasoning benchmarks show that our method consistently improves over representative SFT, distillation, and RL baselines. Our results highlight the importance of coordinating data difficulty across SFT and RL for effective SLM reasoning post-training.
SFT or RL? An Early Investigation into Training R1-Like Reasoning Large Vision-Language Models
University of California, Santa Cruz(加州大学圣克鲁兹分校);University of Texas at Dallas(德克萨斯大学达拉斯分校);The Pennsylvania State University(宾夕法尼亚州立大学);Amazon Research(亚马逊研究院)
Learning to Adapt SFT Data for Better Reasoning Generalization
Beihang University(北京航空航天大学);Tsinghua University(清华大学);Nanyang Technological University(南洋理工大学);Hangzhou International Innovation Institute(杭州国际创新研究院);Beijing Advanced Innovation Center for Future Blockchain and Privacy Computing(北京未来区块链与隐私计算高级创新中心)
Reassessing the Role of Supervised Fine-Tuning: An Empirical Study in VLM Reasoning
NLPR & MAIS, Institute of Automation, Chinese Academy of Sciences(人工智能研究院 & 模式识别与人工智能研究所,中国科学院自动化研究所);School of Artificial Intelligence, University of Chinese Academy of Sciences(人工智能学院,中国科学院大学);Meituan(美团)
State Key Laboratory of AI Safety, Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China(人工智能安全国家重点实验室,计算技术研究所,中国科学院,北京);University of Chinese Academy of Sciences, Beijing, China(中国科学院大学,北京)
RLKD: Distilling LLMs' Reasoning via Reinforcement Learning
State Key Laboratory of AI Safety, Institute of Computing Technology, CAS(人工智能安全国家重点实验室,计算技术研究所,中国科学院);University of Chinese Academy of Sciences(中国科学院大学);Huawei Inc.(华为公司)
RASFT: Rollout-Adaptive Supervised Fine-Tuning for Reasoning
The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳));Shanghai Artificial Intelligence Laboratory(上海人工智能实验室);New Jersey Institute of Technology(新泽西理工学院);Institute of Computing Technology, CAS(中国科学院计算技术研究所)
Rethinking Expert Trajectory Utilization in LLM Post-training for Mathematical Reasoning
Zhejiang University(浙江大学);School of Engineering, Westlake University(西湖大学工程学院);Institute of Advanced Technology, Westlake Institute for Advanced Study(西湖先进研究院技术研究所);Huawei Noah’s Ark Lab(华为诺亚实验室)
Enhancing Large Language Model Reasoning via Selective Critical Token Fine-Tuning
Southern University of Science and Technology(南方科技大学);Peking University(北京大学);Shanghai University of Finance and Economics(上海金融学院);Tsinghua University(清华大学)
Teaching Thinking Models to Reason with Tools: A Full-Pipeline Recipe for Tool-Integrated Reasoning
Zhejiang University(浙江大学);Shanghai AI Laboratory(上海人工智能实验室);Peking University(北京大学);The Chinese University of Hong Kong(香港中文大学);Tsinghua University(清华大学);University of Science and Technology of China(中国科学技术大学);Shanghai Jiao Tong University(上海交通大学)
Learning When to Reason for Text-to-SQL via SFT and DPO
Department of Electrical and Computer Engineering, Seoul National University(首尔国立大学电气与计算机工程系);AI Center, Samsung Electronics(三星电子人工智能中心);AIIS, ASRI, INMC, ISRC, and IPAI, Seoul National University(首尔国立大学人工智能研究所、高级科学研究所以及综合纳米技术中心、信息存储研究中心和人工智能平台研究所)
Beyond SFT-to-RL: Pre-alignment via Black-Box On-Policy Distillation for Multimodal RL
Hong Kong University of Science and Technology (Guangzhou)(香港理工大学(广州));Tsinghua University(清华大学);Nanyang Technological University(南洋理工大学);Renmin University of China(中国人民大学);University of Science and Technology of China(中国科学技术大学);University of Chinese Academy of Sciences(中国科学院大学)
Learning from Mistakes: Negative Reasoning Samples Enhance Out-of-Domain Generalization
CAS Key Laboratory of AI Safety, Institute of Computing Technology, CAS, Beijing, China(中国科学院人工智能安全重点实验室,计算技术研究所,中国科学院,北京,中国);Harbin Institute of Technology, Harbin, China(哈尔滨工业大学,哈尔滨,中国);University of Chinese Academy of Sciences, Beijing, China(中国科学院大学,北京,中国);Tsinghua University, Beijing, China(清华大学,北京,中国)