Diffusion language models (DLMs) enable parallel, order-agnostic generation with iterative refinement, offering a flexible alternative to autoregressive large language models (LLMs). However, adapting reinforcement learning (RL) fine-tuning to DLMs remains an open challenge because of the intractable likelihood. Pioneering work such as diffu-GRPO estimated token-level likelihoods via one-step unmasking. While computationally efficient, this approach is severely biased. A more principled foundation lies in sequence-level likelihoods, where the evidence lower bound (ELBO) serves as a surrogate. Yet, despite this clean mathematical connection, ELBO-based methods have seen limited adoption due to the prohibitive cost of likelihood evaluation. In this work, we revisit ELBO estimation and disentangle its sources of variance. This decomposition motivates reducing variance through fast, deterministic integral approximations along a few pivotal dimensions. Building on this insight, we introduce Group Diffusion Policy Optimization (GDPO), a new RL algorithm tailored for DLMs. GDPO leverages simple yet effective Semi-deterministic Monte Carlo schemes to mitigate the variance explosion of ELBO estimators under vanilla double Monte Carlo sampling, yielding a provably lower-variance estimator under tight evaluation budgets. Empirically, GDPO achieves consistent gains over pretrained checkpoints and outperforms diffu-GRPO, one of the state-of-the-art baselines, on the majority of math, reasoning, and coding benchmarks.
Teacher-Guided Policy Optimization for On-Policy Reasoning Distillation under Large Policy Divergence
School of Computer Science and Engineering, Northeastern University, China（东北大学计算机科学与工程学院）；Tsinghua University（清华大学）；Meituan（美团）；Meta AI；NiuTrans Research, Shenyang, China（新译研究院，沈阳，中国）
Slow-Fast Policy Optimization: Reposition-Before-Update for LLM Reasoning
Georgia Institute of Technology（佐治亚理工学院）；University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；University of Manchester（曼彻斯特大学）；NVIDIA；Independent（独立）；Emory University（埃默里大学）
wd1: Weighted Policy Optimization for Reasoning in Diffusion Language Models
Department of Statistical Science, University College London, United Kingdom（伦敦大学统计科学系）；UCL AI Centre, University College London, United Kingdom（伦敦大学UCL人工智能中心）；Graduate School of AI, Ulsan National Institute of Science and Technology, South Korea（韩国乌山国立科学与技术研究院人工智能研究生院）；Department of Mathematics and Computer Science, Universität Basel, Switzerland（巴塞尔大学数学与计算机科学系）
ORBIT: On-policy Exploration-Exploitation for Controllable Multi-Budget Reasoning
School of Computer Science, Peking University（北京大学计算机科学学院）；National Key Laboratory for Multimedia Information Processing, Peking University（北京大学多媒体信息处理国家重点实验室）；LLM Department, Tencent（腾讯LLM部门）；The Hong Kong University of Science and Technology（香港科技大学）
Structured Role-Aware Policy Optimization for Multimodal Reasoning
School of Computing & Data Science, The University of Hong Kong（计算与数据科学学院，香港大学）；School of Computing & Data Science and Institute of Data Science, The University of Hong Kong（计算与数据科学学院和数据科学研究所，香港大学）
RASFT: Rollout-Adaptive Supervised Fine-Tuning for Reasoning
The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；New Jersey Institute of Technology（新泽西理工学院）；Institute of Computing Technology, CAS（中国科学院计算技术研究所）
C$^2$GSPG: Confidence-calibrated Group Sequence Policy Gradient towards Self-aware Reasoning
Gaoling School of Artificial Intelligence, Renmin University of China（中国人民大学高陵人工智能学院）；Tsinghua University（清华大学）；Beijing Key Laboratory of Research on Large Models（北京大模型研究关键实验室）；Engineering Research Center of Next-Generation Intelligent Search（下一代智能搜索工程研究中心）
Wharton School, University of Pennsylvania（宾夕法尼亚大学沃顿商学院）；University of Pennsylvania（宾夕法尼亚大学）；Graduate Group in Applied Mathematics and Computational Science, University of Pennsylvania（宾夕法尼亚大学应用数学与计算科学联合组）
Towards Generalizable Reasoning: Group Causal Counterfactual Policy Optimization for LLM Reasoning
Institute of Software, Chinese Academy of Sciences（中国科学院软件研究所）；University of Chinese Academy of Sciences（中国科学院大学）；Wangxuan Institute of Computer Technology, Peking University（北京大学王宣计算机技术研究所）；The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））
