On-policy distillation (OPD) consolidates expert capabilities into a single student by matching teacher and student final-layer distributions on student-generated rollouts. We investigate whether depthwise prediction transitions, the changes in predictions between layers, can provide additional task-relevant supervision. Our diagnostic study finds that stronger teacher--student agreement in intermediate transitions is associated with higher answer correctness within questions, with correct responses exhibiting lower mean transition KL across all measured intermediate layer pairs. These observations motivate On-Policy Transition Distillation (OPTD), which augments final-layer distribution matching by aligning teacher and student layer-to-layer log-probability increments on student-generated rollouts. Evaluations across three backbones and eight benchmarks spanning mathematics, coding, and instruction following show that OPTD achieves the best or tied-best overall performance in both dual- and three-domain distillation. In the three-domain setting, OPTD recovers $\approx 90\%$ of the overall teacher--student capability gap, compared with $\approx 75\%$ for OPD.
Step-Level On-Policy Distillation: Interpolating Between On-Policy Distillation and Supervised Fine-Tuning
State Key Laboratory for Novel Software Technology, Nanjing University(南京大学现代软件技术国家重点实验室);XingYun Lab, HUJING Digital Media & Entertainment Group(星云实验室,沪景数字媒体娱乐集团);University of Chinese Academy of Sciences(中国科学院大学);School of Data Science, Fudan University(复旦大学数据科学学院)
Department of Computer Science(计算机科学系);Engineering, Shanghai Jiao Tong University(上海交通大学工程学院);Shanghai Innovation Institute(上海创新研究院);Large Language Department, Tencent(腾讯大语言部门)
SCOPE: Signal-Calibrated On-Policy Distillation Enhancement with Dual-Path Adaptive Weighting
University of Science and Technology of China(中国科学技术大学);Meituan LongCat Interaction Team(美团 LongCat 交互团队);Nanjing University(南京大学);Fudan University(复旦大学);Huazhong University of Science and Technology(华中科技大学)
Influence-Directed Distillation: Solving the Diversity Bottleneck in Sampled-Token On-Policy Distillation
University of North Carolina at Chapel Hill(北卡罗来纳大学教堂山分校);University of Science and Technology of China(中国科学技术大学);Shanghai University of Finance and Economics(上海财经大学)
Beyond the Teacher: Understanding and Preserving Distillation Overshoot in Agentic On-Policy Distillation
University of Southern California(南加州大学);University of Florida(佛罗里达大学);Emory University(埃默里大学);University of Alabama(阿拉巴马大学);University of Maryland(马里兰大学);Amazon AGI(亚马逊AGI);Boston University(波士顿大学);The Ohio State University(俄亥俄州立大学);National Taiwan University(台湾大学)
Entropy-Aware On-Policy Distillation of Language Models
IBM Research, San Jose, CA, USA(IBM研究院,旧金山,加州,美国);University of Toronto, Ontario, Canada(多伦多大学,安大略,加拿大);Vector Institute, Ontario, Canada(向量研究所,安大略,加拿大)
SOD: Step-wise On-policy Distillation for Small Language Model Agents
Zhejiang University(浙江大学);Large Language Model Department, Tencent(腾讯大语言模型部门);University of Science and Technology of China(中国科学技术大学);National University of Singapore(新加坡国立大学)
On-Policy Distillation for Vision-Language Model Adaptation, an Effective Paradigm on Low-Quality Multimodal Data
The University of Hong Kong(香港大学);Great Wall Motor(长城汽车);Wuhan University(武汉大学);China University of Petroleum (East China)(中国石油大学(华东));University of Science and Technology of China(中国科学技术大学);Northwestern Polytechnical University(西北工业大学)
SEED: Self-Evolving On-Policy Distillation for Agentic Reinforcement Learning
Tsinghua University(清华大学);Zhejiang University(浙江大学);The Chinese University of Hong Kong(香港中文大学);Nanyang Technological University(南洋理工大学);Tongji University(同济大学)
Learning from Repaired Reasoning: Root-Cause-Guided On-Policy Distillation
Gaoling School of Artificial Intelligence, Renmin University of China(中国人民大学高瓴人工智能学院);School of Software and Microelectronics, Peking University(北京大学软件与微电子学院);School of Information Technology and Management, University of International Business and Economics(对外经济贸易大学信息与技术管理学院)