Efficient reasoning distillation for long chain-of-thought (CoT) models is increasingly constrained by the assumption of a single oracle teacher, despite the practical availability of multiple candidate teachers and growing CoT corpora. We revisit teacher selection and observe that different students have different "best teachers," and even for the same student, the best teacher can vary across datasets. Therefore, to unify multiple teachers' reasoning abilities into a student to overcome conflicts among various teachers' supervision, we propose Merge-of-Thought Distillation (MoT), a lightweight framework that alternates between teacher-specific supervised fine-tuning branches and weight-space merging of the resulting student variants. On competition math benchmarks, using only about 200 CoT samples, applying MoT to a Qwen3-14B student surpasses strong models including Deepseek-R1, Qwen3-32B, and OpenAI-O1, demonstrating substantial gains. Besides, MoT consistently outperforms the best single-teacher distillation, improves general reasoning beyond mathematics while reducing catastrophic forgetting, and shows robustness to distribution-shifted and peer-level teachers. Finally, we have demonstrated MoT possesses consensus CoT by eliminating teacher-specific inductive biases and inter-teacher conflicts while repeatedly reinforcing the learning of consensus reasoning features. These results position MoT as a simple, effective route to efficiently distilling long CoT capabilities from diverse teachers into compact students.
Symbolic Chain-of-Thought Distillation: Small Models Can Also "Think" Step-by-Step
University of California Los Angeles（加利福尼亚大学洛杉矶分校）；Allen Institute for Artificial Intelligence（艾伦人工智能研究所）；University of Southern California（南加利福尼亚大学）；Yonsei University（延世大学）；University of Washington（华盛顿大学）
CoT2Align: Cross-Chain of Thought Distillation via Optimal Transport Alignment for Language Models with Different Tokenizers
Hanoi University of Science and Technology（河内理工大学）；Oraichain Labs Inc.（Oraichain实验室公司）；ByteDance Inc（字节跳动公司）；VNU University of Engineering and Technology（越南国家大学工程技术大学）；Monash University（莫纳什大学）；University of Oregon（俄勒冈大学）
Pre-trained multilingual language models (LMs) have achieved state-of-the-art results in cross-lingual transfer, but they often lead to an inequitable representation of languages due to limited capacity, skewed pre-training data, and sub-optimal vocabularies. This has prompted the creation of an ever-growing pre-trained model universe, where each model is trained on large amounts of language or domain specific data with a carefully curated, linguistically informed vocabulary. However, doing so brings us back full circle and prevents one from leveraging the benefits of multilinguality. To addre…
Key-Point-Driven Mathematical Reasoning Distillation of Large Language Model
Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）；School of Cyber Security, University of Chinese Academy of Sciences（中国科学院大学网络空间安全学院）；School of Artificial Intelligence, Beijing Normal University（北京师范大学人工智能学院）
Beyond Imitation: Learning Key Reasoning Steps from Dual Chain-of-Thoughts in Reasoning Distillation
Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）；School of Cyber Security, University of Chinese Academy of Sciences（中国科学院大学网络空间安全学院）
When Compression Helps and When It Hurts: Condition-Aware Analysis of Chain-of-Thought Distillation
Ningbo Institute of Digital Twin, Eastern Institute of Technology, Ningbo（宁波数字孪生研究院，东方理工高等研究院，宁波）；Viterbi School of Engineering, University of Southern California（南加州大学维特比工程学院）；The Hong Kong Polytechnic University（香港理工大学）；LMU Munich（慕尼黑大学）；Saarland University（萨尔大学）
Distilling Mathematical Reasoning Capabilities into Small Language Models
Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）；School of Cyber Security, University of Chinese Academy of Sciences（中国科学院大学网络空间安全学院）；Gaoling School of Artificial Intelligence, Renmin University of China（中国人民大学高瓴人工智能学院）
TT-XAI: Trustworthy Clinical Text Explanations via Keyword Distillation and LLM Reasoning
Faculty of Computer and Information Science, University of Ljubljana, Slovenia（卢布尔雅那大学计算机与信息科学学院）；ICAM - Advanced Environmental Research Institute, West University of Timisoara, Romania（蒂米șoara西大学先进环境研究所）；Department of Computer Science, West University of Timisoara, Romania（蒂米șoa拉西大学计算机科学系）
Unveiling the Key Factors for Distilling Chain-of-Thought Reasoning
Department of Computing, The Hong Kong Polytechnic University（香港理工大学计算机系）；Ningbo Key Laboratory of Spatial Intelligence and Digital Derivative（宁波空间智能与数字衍生关键实验室）；Institute of Digital Twin, EIT（数字孪生研究院）；Saarland University（萨尔兰大学）；Meituan Inc.（美团公司）
Improving Reasoning Capabilities in Small Models through Mixture-of-Layers Distillation with Stepwise Attention on Key Information
Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）；School of Cyber Security, University of Chinese Academy of Sciences（中国科学院大学网络安全学院）
