Department of Statistics and Data Science, Southern University of Science and Technology(统计与数据科学系,南方科技大学);Department of Computer Sciences, University of Wisconsin–Madison(计算机科学系,威斯康星大学麦迪逊分校)
推理蒸馏已成为将大推理模型的能力转移到小语言模型的主流范式。然而,推理蒸馏存在数据污染风险:基准数据可能意外包含在蒸馏数据中,从而夸大模型性能指标。本文正式定义了蒸馏数据检测任务,确定给定问题是否包含在模型的蒸馏数据中。该任务的独特挑战在于蒸馏数据的不完全可用性。为此,我们提出Token Probability Deviation(TPD),一种利用模型生成的输出token概率模式而非输入token的概率检测方法。我们的方法受观察启发:已见的问题倾向于生成更多近确定性的token,而未见的问题则不然。因此,TPD分数被设计用来量化生成token与高置信度参考概率的token层面偏差。因此,已见的问题会产生显著较低的TPD分数,从而实现强大的检测性能。广泛的实验展示了我们方法的有效性,在蒸馏数据集上将检测AUC提高高达31%。
英文摘要:
Reasoning distillation has emerged as a prevailing paradigm for transferring reasoning capabilities from large reasoning models to small language models. Yet, reasoning distillation risks data contamination: benchmark data may inadvertently be included in the distillation data, thereby inflating model performance metrics. In this work, we formally define the distillation data detection task, which determines whether a given question is included in the model's distillation data. The unique challenge of this task lies in the partial availability of distillation data. To address this, we propose Token Probability Deviation (TPD), a detection method that leverages the probability patterns of output tokens generated by the model instead of input tokens. Our method is motivated by the observation that seen questions tend to elicit more near-deterministic tokens generated by the models than unseen ones. Our TPD score is thus designed to quantify the token-level deviation of generated tokens from a high-confidence reference probability. Consequently, seen questions can yield substantially lower TPD scores than unseen ones, enabling strong detection performance. Extensive experiments demonstrate the effectiveness of our approach, improving detection AUC by up to 31% on distillation datasets.
Backtracking When It Strays: Mitigating Dual Exposure Biases in LLM Reasoning Distillation
College of Computer Science and Technology, Jilin University(吉林大学计算机科学与技术学院);Key Laboratory of Symbolic Computation and Knowledge Engineering, MoE, Jilin University(吉林大学符号计算与知识工程重点实验室);Tongyi Lab, Alibaba Group(阿里集团通义实验室);College of Computer Science and Technology, Zhejiang University(浙江大学计算机科学与技术学院);School of Artificial Intelligence, Jilin University(吉林大学人工智能学院)
Key-Point-Driven Mathematical Reasoning Distillation of Large Language Model
Institute of Information Engineering, Chinese Academy of Sciences(中国科学院信息工程研究所);School of Cyber Security, University of Chinese Academy of Sciences(中国科学院大学网络空间安全学院);School of Artificial Intelligence, Beijing Normal University(北京师范大学人工智能学院)
Hán Dān Xué Bù (Mimicry) or Qīng Chū Yú Lán (Mastery)? A Cognitive Perspective on Reasoning Distillation in Large Language Models
School of Philosophy, Anhui University(安徽大学哲学学院);Institute of Neuroscience, Chinese Academy of Sciences(中国科学院神经科学研究所);Faculty of Education, University of Cambridge(剑桥大学教育学院);School of Foreign Studies, South China Normal University(华南师范大学外语学院);Division of Psychology and Language Sciences, University College London(伦敦大学学院心理学与语言科学系)
Improve Student's Reasoning Generalizability through Cascading Decomposed CoTs Distillation
Institute of Information Engineering, Chinese Academy of Sciences(中国科学院信息工程研究所);School of Cyber Security, University of Chinese Academy of Sciences(中国科学院大学网络空间安全学院)
Improving Reasoning Capabilities in Small Models through Mixture-of-Layers Distillation with Stepwise Attention on Key Information
Institute of Information Engineering, Chinese Academy of Sciences(中国科学院信息工程研究所);School of Cyber Security, University of Chinese Academy of Sciences(中国科学院大学网络安全学院)
Beyond Imitation: Learning Key Reasoning Steps from Dual Chain-of-Thoughts in Reasoning Distillation
Institute of Information Engineering, Chinese Academy of Sciences(中国科学院信息工程研究所);School of Cyber Security, University of Chinese Academy of Sciences(中国科学院大学网络空间安全学院)
Learning from Repaired Reasoning: Root-Cause-Guided On-Policy Distillation
Gaoling School of Artificial Intelligence, Renmin University of China(中国人民大学高瓴人工智能学院);School of Software and Microelectronics, Peking University(北京大学软件与微电子学院);School of Information Technology and Management, University of International Business and Economics(对外经济贸易大学信息与技术管理学院)
Where Did This Sentence Come From? Tracing Provenance in LLM Reasoning Distillation
College of Computer Science and Technology(计算机科学与技术学院);Alibaba Cloud Computing(阿里云计算);School of Artificial Intelligence(人工智能学院);Department of Mathematics(数学系)
LLM-Guided Knowledge Distillation for Temporal Knowledge Graph Reasoning
School of Computer Science and Technology, Xidian University(西安电子科技大学计算机科学与技术学院);School of Computing and Artificial Intelligence, Southwest Jiaotong University(西南交通大学计算机与人工智能学院);School of Information Science and Engineering, Chongqing Jiaotong University(重庆交通大学信息科学与工程学院)