While the shift from cascaded dialogue systems to end-to-end (E2E) speech Large Language Models (LLMs) improves latency and paralinguistic modeling, E2E models often exhibit a significant performance degradation compared to their text-based counterparts. The standard Supervised Fine-Tuning (SFT) and Reinforcement Learning (RL) training methods fail to close this gap. To address this, we propose X-OPD, a novel Cross-Modal On-Policy Distillation framework designed to systematically align the capabilities of Speech LLMs to their text-based counterparts. X-OPD enables the Speech LLM to explore its own distribution via on-policy rollouts, where a text-based teacher model evaluates these trajectories and provides token-level feedback, effectively distilling teacher's capabilities into student's multi-modal representations. Extensive experiments across multiple benchmarks demonstrate that X-OPD significantly narrows the gap in complex tasks while preserving the model's inherent capabilities.
BAS-OPD: Budget-Aware Selective On-Policy Self-Distillation for Fine-Grained Multimodal Perception
HFIPS, Chinese Academy of Sciences（中国科学院合肥物质科学研究院）；University of Science and Technology of China（中国科学技术大学）；University of California, Los Angeles（加利福尼亚大学洛杉矶分校）
Sequential Beats Joint: On the Interplay between On-Policy Distillation and RLVR
New York University（纽约大学）；University of Chicago（芝加哥大学）；University of Waterloo（滑铁卢大学）；University of Alberta（阿尔伯塔大学）；NYU Shanghai（纽约大学上海分校）；Alberta Machine Intelligence Institute (Amii)（阿尔伯塔机器智能研究所）
Learning to Revise Reasoning with Segment-wise On-Policy Distillation
The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；University of Science and Technology of China（中国科学技术大学）；Alibaba Group（阿里巴巴集团）；Stanford University（斯坦福大学）
Decoupling KL and Trajectories: A Unified Perspective for SFT, DAgger, Offline RL, and OPD in LLM Distillation
Eastern Institute of Technology（东技术院）；The Hong Kong Polytechnic University（香港理工大学）；Shanghai Jiao Tong University（上海交通大学）；Thrust of Artificial Intelligence, The Hong Kong University of Science and Technology (Guangzhou)（人工智能 thrust，香港科学与技术大学（广州））
Draft-OPD: On-Policy Distillation for Speculative Draft Models
Shanghai Jiao Tong University（上海交通大学）；Shanghai AI Laboratory（上海人工智能实验室）；Tsinghua University（清华大学）；The Chinese University of Hong Kong（香港中文大学）；Peking University（北京大学）；Zhejiang University（浙江大学）
OPD-V: Visual On-Policy Self-Distillation with Modality Balance
National University of Singapore（新加坡国立大学）；Ludwig Maximilian University of Munich（慕尼黑大学）；Munich Center for Machine Learning（慕尼黑机器学习中心）；Sun Yat-sen University（中山大学）
Rethinking On-Policy Distillation of Large Language Models II: One Training Example
University of Chinese Academy of Sciences（中国科学院大学）；Tsinghua University（清华大学）；Northeastern University（东北大学）；University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；Johns Hopkins University（约翰斯·霍普金斯大学）
Demystifying OPD: Length Inflation and Stabilization Strategies for Large Language Models
Department of Computer Science, Rice University, Houston, USA（莱斯大学计算机科学系，美国休斯顿）；Department of Computer Science, University of North Carolina at Charlotte, Charlotte, USA（北卡罗来纳大学夏洛特分校计算机科学系，美国夏洛特）；Department of Computer Science, Johns Hopkins University, Baltimore, USA（约翰霍普金斯大学计算机科学系，美国巴尔的摩）；Department of Computer and Data Sciences, Case Western Reserve University（凯斯西储大学计算机与数据科学系）
Flow-OPD: On-Policy Distillation for Flow Matching Models
University of Science and Technology of China（中国科学技术大学）；University of California, Los Angeles（加州大学洛杉矶分校）；The Chinese University of Hong Kong（香港中文大学）；Xiaohongshu Inc.（小红书公司）
Do We Really Need KL Divergence for On-Policy Distillation of Large Language Models?
LeapLab, Tsinghua University（清华大学LeapLab）；Qiuzhen College, Tsinghua University（清华大学求真书院）；Beihang University（北京航空航天大学）；National University of Singapore（新加坡国立大学）；The Chinese University of Hong Kong（香港中文大学）；E Fund Management Co., Ltd.（易方达基金管理有限公司）
Revisiting On-Policy Distillation: Empirical Failure Modes and Simple Fixes
State Key Laboratory of Multimodal Artificial Intelligence Systems, CASIA（多模态人工智能系统国家重点实验室，中科院自动化所）；School of Artificial Intelligence, UCAS（中国科学技术大学人工智能学院）；Fudan University（复旦大学）；Independent Researcher（独立研究者）
Every Coin Has Two Sides: On the Dual Nature of Generalization in On-Policy Distillation of Large Language Models
University of Science and Technology of China（中国科学技术大学）；Peking University（北京大学）；IQuest Research（IQuest研究院）；MBZUAI（穆罕默德·本·扎耶德人工智能大学）；Zhejiang University（浙江大学）
Towards Understanding On-Policy Distillation through the Lens of Test-Time Scaling
Hong Kong Baptist University（香港浸会大学）；University of Texas at Austin（德克萨斯大学奥斯汀分校）；Shanghai Jiao Tong University（上海交通大学）；Shanghai Innovation Institute（上海创新研究院）；Ant Group（蚂蚁集团）
Dense Is Not Enough: Hierarchical Supervision Allocation for Long-Horizon On-Policy Distillation
Ant Group（蚂蚁集团）；Alibaba International Digital Commerce Group（阿里巴巴国际数字商业集团）；University of Science and Technology of China（中国科学技术大学）；Peking University（北京大学）；University of Electronic Science and Technology of China（电子科技大学）
PowerOPD: Stabilizing On-Policy Distillation with Bounded Power Transformation
Eastern Institute of Technology, Ningbo（宁波东方理工大学）；The Hong Kong Polytechnic University（香港理工大学）；Shanghai Jiao Tong University（上海交通大学）；University of Waterloo（滑铁卢大学）
Teacher-Guided Policy Optimization for On-Policy Reasoning Distillation under Large Policy Divergence
School of Computer Science and Engineering, Northeastern University, China（东北大学计算机科学与工程学院）；Tsinghua University（清华大学）；Meituan（美团）；Meta AI；NiuTrans Research, Shenyang, China（新译研究院，沈阳，中国）
