Recent works have shown that chain-of-thought (CoT) prompting can elicit language models to solve complex reasoning tasks, step-by-step. However, prompt-based CoT methods are dependent on very large models such as GPT-3 175B which are prohibitive to deploy at scale. In this paper, we use these large models as reasoning teachers to enable complex reasoning in smaller models and reduce model size requirements by several orders of magnitude. We propose Fine-tune-CoT, a method that generates reasoning samples from very large teacher models to fine-tune smaller models. We evaluate our method on a wide range of public models and complex tasks. We find that Fine-tune-CoT enables substantial reasoning capability in small models, far outperforming prompt-based baselines and even the teacher model in many tasks. Additionally, we extend our method by leveraging the teacher model's ability to generate multiple distinct rationales for each original sample. Enriching the fine-tuning data with such diverse reasoning results in a substantial performance boost across datasets, even for very small models. We conduct ablations and sample studies to understand the emergence of reasoning capabilities of student models. Our code implementation and data are available at https://github.com/itsnamgyu/reasoning-teacher.
Teacher-Guided Policy Optimization for On-Policy Reasoning Distillation under Large Policy Divergence
School of Computer Science and Engineering, Northeastern University, China（东北大学计算机科学与工程学院）；Tsinghua University（清华大学）；Meituan（美团）；Meta AI；NiuTrans Research, Shenyang, China（新译研究院，沈阳，中国）
More Than One Teacher: Adaptive Multi-Guidance Policy Optimization for Diverse Exploration
Tongji University（同济大学）；Hong Kong Polytechnic University（香港理工大学）；Shanghai AI Lab（上海人工智能实验室）；National University of Singapore（新加坡国立大学）；University of Electronic Science and Technology of China（电子科技大学）
"The Whole Is Greater Than the Sum of Its Parts": A Compatibility-Aware Multi-Teacher CoT Distillation Framework
State Key Laboratory of Human-Machine Hybrid Augmented Intelligence, Institute of Artificial Intelligence and Robotics, Xi’an Jiaotong University（人机混合增强智能国家重点实验室，人工智能与机器人研究院，西安交通大学）；Nankai University（南开大学）；The Hong Kong University of Science and Technology(Guangzhou)（香港科技大学（广州））；School of Software Engineering, State Key Laboratory of Human-Machine Hybrid Augmented Intelligence, Institute of Artificial Intelligence and Robotics, Xi’an Jiaotong University（软件学院，人机混合增强智能国家重点实验室，人工智能与机器人研究院，西安交通大学）
Skill-Conditioned Gated Self-Distillation for LLM Reasoning
Tsinghua University（清华大学）；Fudan University（复旦大学）；City University of Hong Kong（香港城市大学）；Huazhong University of Science and Technology（华中科技大学）；University of Wisconsin-Madison（威斯康星大学麦迪逊分校）
Reasoning Compression with Mixed-Policy Distillation
Technical University of Munich（慕尼黑技术大学）；GESIS – Leibniz Institute for the Social Sciences（莱布尼茨社会科学研究所）；Northeastern University（东北大学）；LMU Munich（慕尼黑大学）；Siemens AG（西门子股份公司）；University of Cambridge（剑桥大学）；University of Oxford（牛津大学）；Mina AI
Learning to Revise Reasoning with Segment-wise On-Policy Distillation
The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；University of Science and Technology of China（中国科学技术大学）；Alibaba Group（阿里巴巴集团）；Stanford University（斯坦福大学）
Audit Before You Distill: A Controlled Workflow for Reasoning Transfer into Compact Medical Language Models
Einstein Global Advanced Technologies for Equity (E-GATE), Hospital Israelita Albert Einstein（以色列阿尔伯特·爱因斯坦医院 爱因斯坦全球公平先进技术中心（E-GATE））；Institute of Science and Technology Itaú (ICTi), Itaú Unibanco（伊塔乌联合银行 伊塔乌科学技术研究所（ICTi））；School of Sciences, São Paulo State University（圣保罗州立大学 理学院）
Where Did This Sentence Come From? Tracing Provenance in LLM Reasoning Distillation
College of Computer Science and Technology（计算机科学与技术学院）；Alibaba Cloud Computing（阿里云计算）；School of Artificial Intelligence（人工智能学院）；Department of Mathematics（数学系）
From Atoms to Chains: Divergence-Guided Reasoning Curriculum for Unlabeled LLM Domain Adaptation
Beijing Key Laboratory of Intelligent Information Technology, School of Computer Science and Technology, Beijing Institute of Technology（北京智能信息科技重点实验室，计算机科学与技术学院，北京理工大学）；ByteDance China（字节跳动中国）
RLKD: Distilling LLMs' Reasoning via Reinforcement Learning
State Key Laboratory of AI Safety, Institute of Computing Technology, CAS（人工智能安全国家重点实验室，计算技术研究所，中国科学院）；University of Chinese Academy of Sciences（中国科学院大学）；Huawei Inc.（华为公司）
Improving Reasoning Capabilities in Small Models through Mixture-of-Layers Distillation with Stepwise Attention on Key Information
Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）；School of Cyber Security, University of Chinese Academy of Sciences（中国科学院大学网络安全学院）
Backtracking When It Strays: Mitigating Dual Exposure Biases in LLM Reasoning Distillation
College of Computer Science and Technology, Jilin University（吉林大学计算机科学与技术学院）；Key Laboratory of Symbolic Computation and Knowledge Engineering, MoE, Jilin University（吉林大学符号计算与知识工程重点实验室）；Tongyi Lab, Alibaba Group（阿里集团通义实验室）；College of Computer Science and Technology, Zhejiang University（浙江大学计算机科学与技术学院）；School of Artificial Intelligence, Jilin University（吉林大学人工智能学院）
Long-Chain Reasoning Distillation via Adaptive Prefix Alignment
School of Computer Science and Engineering, Northeastern University, China（东北大学计算机科学与工程学院）；Department of Computer Science and Technology, Institute for AI, Tsinghua University, China（清华大学计算机科学与技术系）；Alibaba Group, China（阿里巴巴集团）
Enhancing Long-Chain Reasoning Distillation through Error-Aware Self-Reflection
School of Computer Science and Engineering, Northeastern University, China（东北大学计算机科学与工程学院）；Department of Computer Science and Technology, Institute for AI, Tsinghua University, China（清华大学人工智能研究院计算机科学与技术系）；School of Computer Science, Beijing University of Posts and Telecommunications, China（北京邮电大学计算机科学学院）
