提出Rank-Surprisal Ratio (RSR)度量,通过结合对齐性和信息性评估推理轨迹对学生模型的适用性,在轨迹选择和教师选择中显著优于现有方法。
摘要:
长链思维(CoT)轨迹为从教师到学生大语言模型的推理蒸馏提供了丰富的监督信号。然而,先前工作和我们的实验均表明,来自更强教师的轨迹并不一定能产生更好的学生,凸显了蒸馏中数据-学生适配性的重要性。现有方法主要通过学生似然评估适配性,倾向于选择与学生模型当前行为高度一致的轨迹,但忽略了更具信息性的轨迹。针对这一问题,我们提出Rank-Surprisal Ratio (RSR),一个简单的度量,同时捕捉对齐性和信息性以评估推理轨迹的适用性。RSR的动机源于有效轨迹通常通过结合低绝对概率和相对高排名的token(在学生模型下)来平衡学习信号强度和行为对齐。具体而言,RSR定义为轨迹的平均token级排名与其平均负对数似然之比,计算和解释直观。在五个学生模型和来自11个不同教师的推理轨迹上,RSR与训练后推理性能强相关(平均Spearman 0.86),持续优于现有度量。我们进一步展示了其在轨迹选择和教师选择中的实际效用。
英文摘要:
Long chain-of-thought (CoT) trajectories provide rich supervision signals for distilling reasoning from teacher to student LLMs. However, both prior work and our experiments show that trajectories from stronger teachers do not necessarily yield better students, highlighting the importance of data-student suitability in distillation. Existing methods assess suitability primarily through student likelihood, favoring trajectories that align closely with the student model's current behavior but overlooking more informative ones. Addressing this, we propose Rank-Surprisal Ratio (RSR), a simple metric that captures both alignment and informativeness to assess the suitability of a reasoning trajectory. RSR is motivated by the observation that effective trajectories typically balance learning signal strength and behavioral alignment by combining low absolute probability with relatively high-ranked tokens under the student model. Concretely, RSR is defined as the ratio of a trajectory's average token-wise rank to its average negative log-likelihood, and is straightforward to compute and interpret. Across five student models and reasoning trajectories from 11 diverse teachers, RSR strongly correlates with post-training reasoning performance (average Spearman 0.86), consistently outperforming existing metrics. We further demonstrate its practical utility in both trajectory selection and teacher selection.
When the Strongest Teacher Is Not the Best Teacher: Student-Centric Answer Selection
University of Washington(华盛顿大学);University of Texas at Austin(德克萨斯大学奥斯汀分校);University of Southern California(南加州大学);Independent Researcher(独立研究者);National University of Singapore(新加坡国立大学);Microsoft(微软);Google(谷歌);Mohamed bin Zayed University of Artificial Intelligence(穆罕默德·本·扎耶德人工智能大学);Northwestern University(西北大学);Allen Institute for AI (AI2)(人工智能研究院(AI2))
Distantly-Supervised Named Entity Recognition with Adaptive Teacher Learning and Fine-grained Student Ensemble
Huawei Cloud(华为云);School of Software Engineering, Huazhong University of Science and Technology(华中科技大学软件工程学院);Peking University(北京大学);Hubei Key Laboratory of Distributed System Security, Hubei Engineering Research Center on Big Data Security, School of Cyber Science and Engineering, Huazhong University of Science and Technology(华中科技大学网络空间安全学院,湖北省分布式系统安全重点实验室,湖北省大数据安全工程技术研究中心)
Improving the Robustness of Distantly-Supervised Named Entity Recognition via Uncertainty-Aware Teacher Learning and Student-Student Collaborative Learning
National Key Laboratory for Multimedia Information Processing, Peking University(北京大学多媒体信息处理国家重点实验室);School of Software and Microelectronics, Peking University(北京大学软件与微电子学院);Tencent Inc.(腾讯公司);Jiangsu Collaborative Innovation Center for Language Ability, Jiangsu Normal University(江苏省语言能力协同创新中心,江苏师范大学)
In Good GRACEs: Principled Teacher Selection for Knowledge Distillation
Princeton Language and Intelligence(普林斯顿语言与智能研究所);Kempner Institute, Harvard(哈佛凯普纳研究所);Microsoft Research, New York(微软研究院(纽约));University of Pennsylvania(宾夕法尼亚大学)