arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2510.04850cs.CLcs.AI
收藏

从推理模型中检测蒸馏数据

Detecting Distillation Data from Reasoning Models

Department of Statistics and Data Science, Southern University of Science and Technology(统计与数据科学系,南方科技大学);Department of Computer Sciences, University of Wisconsin–Madison(计算机科学系,威斯康星大学麦迪逊分校)

Hengxiang Zhang, Hyeong Kyu Choi, Sharon Li, Hongxin Wei

AI总结:

本文提出基于输出token概率偏差的检测方法,用于识别推理模型蒸馏数据中的问题,通过量化生成token与高置信度参考概率的偏差,提升检测性能,实验表明在蒸馏数据集上AUC提升达31%。

摘要:

推理蒸馏已成为将大推理模型的能力转移到小语言模型的主流范式。然而,推理蒸馏存在数据污染风险:基准数据可能意外包含在蒸馏数据中,从而夸大模型性能指标。本文正式定义了蒸馏数据检测任务,确定给定问题是否包含在模型的蒸馏数据中。该任务的独特挑战在于蒸馏数据的不完全可用性。为此,我们提出Token Probability Deviation(TPD),一种利用模型生成的输出token概率模式而非输入token的概率检测方法。我们的方法受观察启发:已见的问题倾向于生成更多近确定性的token,而未见的问题则不然。因此,TPD分数被设计用来量化生成token与高置信度参考概率的token层面偏差。因此,已见的问题会产生显著较低的TPD分数,从而实现强大的检测性能。广泛的实验展示了我们方法的有效性,在蒸馏数据集上将检测AUC提高高达31%。

英文摘要:

Reasoning distillation has emerged as a prevailing paradigm for transferring reasoning capabilities from large reasoning models to small language models. Yet, reasoning distillation risks data contamination: benchmark data may inadvertently be included in the distillation data, thereby inflating model performance metrics. In this work, we formally define the distillation data detection task, which determines whether a given question is included in the model's distillation data. The unique challenge of this task lies in the partial availability of distillation data. To address this, we propose Token Probability Deviation (TPD), a detection method that leverages the probability patterns of output tokens generated by the model instead of input tokens. Our method is motivated by the observation that seen questions tend to elicit more near-deterministic tokens generated by the models than unseen ones. Our TPD score is thus designed to quantify the token-level deviation of generated tokens from a high-confidence reference probability. Consequently, seen questions can yield substantially lower TPD scores than unseen ones, enabling strong detection performance. Extensive experiments demonstrate the effectiveness of our approach, improving detection AUC by up to 31% on distillation datasets.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑