Knowledge distillation (KD) improves language-model pretraining by matching teacher distributions at corpus prefixes, but provides no direct supervision at contexts induced by student predictions. We introduce PA1-OPD, which augments corpus-path distillation with teacher feedback after one student-sampled transition per selected prefix. This expands the supervised states while retaining next-token prediction and corpus-path KD. Data-anchored roots support parallel sampling, and differentiable tree attention shares prefix computation without changing inference. In two continued-pretraining settings, full coverage improves held-out cross-entropy over KD across C4, PG-19, and Pile at every evaluated checkpoint. In Stage2, selecting the 10% of roots with highest student surprisal adds 48.85% to KD's Macro cross-entropy gain over next-token training. Gains remain when branch supervision excludes successor states already on the corpus path, supporting the value of feedback beyond observed contexts. Additional depth or breadth improves aggregate results at unchanged per-term supervision weight, with benefits varying by corpus. These findings support local student-induced supervision as a complement to pretraining distillation.
Decoupling KL and Trajectories: A Unified Perspective for SFT, DAgger, Offline RL, and OPD in LLM Distillation
Eastern Institute of Technology(东技术院);The Hong Kong Polytechnic University(香港理工大学);Shanghai Jiao Tong University(上海交通大学);Thrust of Artificial Intelligence, The Hong Kong University of Science and Technology (Guangzhou)(人工智能 thrust,香港科学与技术大学(广州))
Revisiting On-Policy Distillation: Empirical Failure Modes and Simple Fixes
State Key Laboratory of Multimodal Artificial Intelligence Systems, CASIA(多模态人工智能系统国家重点实验室,中科院自动化所);School of Artificial Intelligence, UCAS(中国科学技术大学人工智能学院);Fudan University(复旦大学);Independent Researcher(独立研究者)
Prefix Teach, Suffix Fade: Local Teachability Collapse in Strong-to-Weak On-Policy Distillation
College of Computer Science and Technology, Zhejiang University(浙江大学计算机科学与技术学院);Meituan LongCat Team, China(美团LongCat团队,中国);College of Computer Science and Technology, Jilin University(吉林大学计算机科学与技术学院)
Data-Efficient Autoregressive-to-Diffusion Language Models via On-Policy Distillation
Department of Computer Science and Engineering, Texas A&M University(德克萨斯大学阿马尔科分校计算机科学与工程系);Department of Bioinformatics and Systems Medicine, University of Texas Health Science Center at Houston(德克萨斯大学健康科学中心休斯顿分校生物信息学与系统医学系);Department of Electrical and Computer Engineering, Texas A&M University(德克萨斯大学阿马尔科分校电气与计算机工程系)
Learning from Repaired Reasoning: Root-Cause-Guided On-Policy Distillation
Gaoling School of Artificial Intelligence, Renmin University of China(中国人民大学高瓴人工智能学院);School of Software and Microelectronics, Peking University(北京大学软件与微电子学院);School of Information Technology and Management, University of International Business and Economics(对外经济贸易大学信息与技术管理学院)
OnlineQAT: On-Policy Distillation for Ultra-Low-Bit Large Language Models
The Hong Kong Polytechnic University(香港理工大学);Sun Yat-sen University(中山大学);PolyU-Daya Bay Technology and Innovation Research Institute(香港理工大学大亚湾技术创新研究院)
Learning from Teacher Continuations at Student States
University of Illinois at Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);Massachusetts Institute of Technology(麻省理工学院);University of Pennsylvania(宾夕法尼亚大学);University of Washington(华盛顿大学);International Business Machines(国际商业机器公司)
Output Vector Editing for Memorization Mitigation in Large Language Models
Center for Information and Language Processing, LMU Munich(慕尼黑大学语言与信息处理中心);Department of Computer Science, University of Copenhagen(哥本哈根大学计算机科学系);Munich Center for Machine Learning(慕尼黑机器学习中心);Pioneer Centre for AI(人工智能先锋中心)
Escaping the KL Agreement Trap in On-Policy Distillation
The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));The Hong Kong University of Science and Technology(香港科技大学);The Hong Kong Polytechnic University(香港理工大学);Eastern Institute of Technology, Ningbo(宁波东方理工大学)
Your Teacher Can't Help You Here: Combating Supervision Fidelity Decay in On-Policy Distillation
University of Chinese Academy of Sciences(中国科学院大学);Chinese Information Processing Laboratory(中文信息处理实验室);Institute of Software, Chinese Academy of Sciences(中国科学院软件研究所);University of Chinese Academy of Sciences, Beijing, China(中国科学院大学,北京,中国)
VAD: Attributing Visual Evidence for Target Reconstruction in Multimodal On-Policy Distillation
Shanghai Jiao Tong University(上海交通大学);Xiaohongshu Inc.(小红书公司);The Chinese University of Hong Kong(香港中文大学);Zhejiang University(浙江大学);Southeast University(东南大学)