Large language models (LMs) beyond a certain scale, demonstrate the emergent capability of generating free-text rationales for their predictions via chain-of-thought (CoT) prompting. While CoT can yield dramatically improved performance, such gains are only observed for sufficiently large LMs. Even more concerning, there is little guarantee that the generated rationales are consistent with LM's predictions or faithfully justify the decisions. In this work, we propose a faithful knowledge distillation method to learn a small, self-consistent CoT model from a teacher model that is orders of magnitude larger. To form better supervision, we elicit rationales supporting the gold answers from a large LM (teacher) by contrastive decoding, which encourages the teacher to generate tokens that become more plausible only when the answer is considered. To ensure faithful distillation, we use the teacher-generated rationales to learn a student LM with a counterfactual reasoning objective, which prevents the student from ignoring the rationales to make inconsistent predictions. Experiments show that, while yielding comparable end-task performance, our method can generate CoT rationales that are more faithful than baselines do. Further analysis suggests that such a model respects the rationales more when making decisions; thus, we can improve its performance more by refining its rationales.
Symbolic Chain-of-Thought Distillation: Small Models Can Also "Think" Step-by-Step
University of California Los Angeles（加利福尼亚大学洛杉矶分校）；Allen Institute for Artificial Intelligence（艾伦人工智能研究所）；University of Southern California（南加利福尼亚大学）；Yonsei University（延世大学）；University of Washington（华盛顿大学）
CoT2Align: Cross-Chain of Thought Distillation via Optimal Transport Alignment for Language Models with Different Tokenizers
Hanoi University of Science and Technology（河内理工大学）；Oraichain Labs Inc.（Oraichain实验室公司）；ByteDance Inc（字节跳动公司）；VNU University of Engineering and Technology（越南国家大学工程技术大学）；Monash University（莫纳什大学）；University of Oregon（俄勒冈大学）
When Compression Helps and When It Hurts: Condition-Aware Analysis of Chain-of-Thought Distillation
Ningbo Institute of Digital Twin, Eastern Institute of Technology, Ningbo（宁波数字孪生研究院，东方理工高等研究院，宁波）；Viterbi School of Engineering, University of Southern California（南加州大学维特比工程学院）；The Hong Kong Polytechnic University（香港理工大学）；LMU Munich（慕尼黑大学）；Saarland University（萨尔大学）
Key-Point-Driven Mathematical Reasoning Distillation of Large Language Model
Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）；School of Cyber Security, University of Chinese Academy of Sciences（中国科学院大学网络空间安全学院）；School of Artificial Intelligence, Beijing Normal University（北京师范大学人工智能学院）
Beyond Imitation: Learning Key Reasoning Steps from Dual Chain-of-Thoughts in Reasoning Distillation
Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）；School of Cyber Security, University of Chinese Academy of Sciences（中国科学院大学网络空间安全学院）
Unveiling the Key Factors for Distilling Chain-of-Thought Reasoning
Department of Computing, The Hong Kong Polytechnic University（香港理工大学计算机系）；Ningbo Key Laboratory of Spatial Intelligence and Digital Derivative（宁波空间智能与数字衍生关键实验室）；Institute of Digital Twin, EIT（数字孪生研究院）；Saarland University（萨尔兰大学）；Meituan Inc.（美团公司）
Improving Reasoning Capabilities in Small Models through Mixture-of-Layers Distillation with Stepwise Attention on Key Information
Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）；School of Cyber Security, University of Chinese Academy of Sciences（中国科学院大学网络安全学院）
Distilling Mathematical Reasoning Capabilities into Small Language Models
Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）；School of Cyber Security, University of Chinese Academy of Sciences（中国科学院大学网络空间安全学院）；Gaoling School of Artificial Intelligence, Renmin University of China（中国人民大学高瓴人工智能学院）
