Diffusion large language models (dLLMs) generate text by iteratively denoising a masked sequence, offering a parallel alternative to autoregressive models, but eliciting strong reasoning through post-training remains difficult: supervised fine-tuning is off-policy and suffers from exposure bias, while reinforcement learning gives only sparse, sequence-level rewards and is hard to apply without tractable sequence likelihoods. On-policy self-distillation (OPSD) offers a promising alternative, using one model as both student and teacher to provide dense, token-level, on-policy supervision, but its effectiveness hinges on giving the teacher privileged information (PI) - typically an instance-specific ground-truth reference unavailable at inference - so the student ends up distilling a weak PI-free consensus policy that yields little improvement on dLLM reasoning. We introduce dOPSD, which instead derives the teacher's privilege directly from the student's own denoising trajectory, evaluating masked positions using later, more-decoded steps of that same trajectory rather than an external label, so the teacher's advantage emerges from the model's own decoding process; on Dream and LLaDA, dOPSD improves both in-domain math reasoning and out-of-domain code generation, outperforming supervised and on-policy baselines.
Department of Computer Science(计算机科学系);Engineering, Shanghai Jiao Tong University(上海交通大学工程学院);Shanghai Innovation Institute(上海创新研究院);Large Language Department, Tencent(腾讯大语言部门)
SCOPE: Signal-Calibrated On-Policy Distillation Enhancement with Dual-Path Adaptive Weighting
University of Science and Technology of China(中国科学技术大学);Meituan LongCat Interaction Team(美团 LongCat 交互团队);Nanjing University(南京大学);Fudan University(复旦大学);Huazhong University of Science and Technology(华中科技大学)
Teacher-Guided Policy Optimization for On-Policy Reasoning Distillation under Large Policy Divergence
School of Computer Science and Engineering, Northeastern University, China(东北大学计算机科学与工程学院);Tsinghua University(清华大学);Meituan(美团);Meta AI;NiuTrans Research, Shenyang, China(新译研究院,沈阳,中国)
SEED: Self-Evolving On-Policy Distillation for Agentic Reinforcement Learning
Tsinghua University(清华大学);Zhejiang University(浙江大学);The Chinese University of Hong Kong(香港中文大学);Nanyang Technological University(南洋理工大学);Tongji University(同济大学)
Reasoning Compression with Mixed-Policy Distillation
Technical University of Munich(慕尼黑技术大学);GESIS – Leibniz Institute for the Social Sciences(莱布尼茨社会科学研究所);Northeastern University(东北大学);LMU Munich(慕尼黑大学);Siemens AG(西门子股份公司);University of Cambridge(剑桥大学);University of Oxford(牛津大学);Mina AI
Teaching the Way, Not the Answer: Privileged Tutoring Distillation for Multimodal Policy Optimization
Tianjin University(天津大学);Beijing Institute of Technology(北京理工大学);Singapore Management University(新加坡国立大学);University of Chinese Academy of Sciences(中国科学院大学);Xiaomi Inc(小米公司)
Institute of Trustworthy Embodied AI(可信具身人工智能研究院);Fudan University(复旦大学);Shanghai Innovation Institute(上海创新研究院);Ant Group(蚂蚁集团);Zhejiang University(浙江大学);City University of Hong Kong(香港城市大学)
Unifying Group-Relative and Self-Distillation Policy Optimization via Sample Routing
Foundation Model Research Center, Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所基础模型研究中心);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院);National University of Singapore(新加坡国立大学);Tencent(腾讯);Wuhan AI Research(武汉人工智能研究院)
Step-Level On-Policy Distillation: Interpolating Between On-Policy Distillation and Supervised Fine-Tuning
State Key Laboratory for Novel Software Technology, Nanjing University(南京大学现代软件技术国家重点实验室);XingYun Lab, HUJING Digital Media & Entertainment Group(星云实验室,沪景数字媒体娱乐集团);University of Chinese Academy of Sciences(中国科学院大学);School of Data Science, Fudan University(复旦大学数据科学学院)
SOD: Step-wise On-policy Distillation for Small Language Model Agents
Zhejiang University(浙江大学);Large Language Model Department, Tencent(腾讯大语言模型部门);University of Science and Technology of China(中国科学技术大学);National University of Singapore(新加坡国立大学)
On-Policy Distillation for Vision-Language Model Adaptation, an Effective Paradigm on Low-Quality Multimodal Data
The University of Hong Kong(香港大学);Great Wall Motor(长城汽车);Wuhan University(武汉大学);China University of Petroleum (East China)(中国石油大学(华东));University of Science and Technology of China(中国科学技术大学);Northwestern Polytechnical University(西北工业大学)