Diffusion-based large language models offer a non-autoregressive alternative for text generation, but enabling them to perform complex reasoning remains challenging. Reinforcement learning has recently emerged as an effective post-training strategy for improving their performance; however, existing methods rely primarily on outcome-based rewards, which provide no direct supervision over the denoising process and often result in poorly structured reasoning that is difficult to interpret and inconsistently supports the final prediction. To address this limitation, we introduce \emph{denoising process reward}, a process-level reinforcement signal defined over the denoising trajectory of diffusion language models. This reward is obtained by estimating the contribution of intermediate denoising intervals to the final task outcome, encouraging the model to favor reasoning trajectories that consistently guide generation toward correct predictions. We further propose an efficient stochastic estimator that reuses standard training rollouts, enabling practical process-level supervision at scale. Experiments on challenging reasoning benchmarks demonstrate that our approach yields consistent improvements in reasoning stability, interpretability, and overall task performance.
Perception-R1: Advancing Multimodal Reasoning Capabilities of MLLMs via Visual Perception Reward
University of Science and Technology of China(中国科学技术大学);State Key Laboratory of Cognitive Intelligence(认知智能国家重点实验室);iFLYTEK AI Research(iFLYTEK人工智能研究院);Hong Kong University of Science and Technology(香港科学与技术大学)
Evaluating and Improving Cultural Awareness of Reward Models for LLM Alignment
Institute of Computing and Intelligence, Harbin Institute of Technology, Shenzhen, China(计算与智能研究所,哈尔滨工业大学,深圳,中国);Peng Cheng Laboratory, Shenzhen, China(鹏城实验室,深圳,中国)
CleanRM: Reward Modeling from Noisy Feedback with a Few Clean Feedback for RLHF
Zhejiang University(浙江大学);Xiaohongshu Inc(小红书公司);National University of Singapore(新加坡国立大学);New York University(纽约大学);University of Illinois at Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);Mohamed bin Zayed University of Artificial Intelligence(穆罕默德·本·扎耶德人工智能大学)
Using Human Feedback to Fine-tune Diffusion Models without Any Reward Model
Tsinghua Shenzhen International Graduate School, Tsinghua University(清华大学深圳国际研究生院);Department of Automation, Tsinghua University(清华大学自动化系);Parametrix Technology Company Ltd.(参数科技有限公司)
NotaGen: Advancing Musicality in Symbolic Music Generation with Large Language Model Training Paradigms
Central Conservatory of Music(中央音乐学院);University of Rochester(罗切斯特大学);Beijing Flowingtech Ltd.(北京流音科技有限公司);Beihang University(北京航空航天大学);Tsinghua University(清华大学)
Advancing LLM Reasoning Generalists with Preference Trees
Tsinghua University(清华大学);University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);Northeastern University(东北大学);Renmin University of China(中国人民大学);BUPT(北京邮电大学);Tencent(腾讯)
Generating and Evolving Reward Functions for Highway Driving with Large Language Models
The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));Guangdong Provincial Key Lab of Integrated Communication, Sensing and Computation for Ubiquitous Internet of Things(广东省普适物联网集成通信、感知与计算重点实验室)
Directly Aligning the Full Diffusion Trajectory with Fine-Grained Human Preference
School of Science and Engineering, The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳)科学与工程学院);Shenzhen International Graduate School, Tsinghua University(清华大学深圳国际研究生院)
Steering Masked Discrete Diffusion Models via Discrete Denoising Posterior Prediction
Université de Montréal(蒙特利尔大学);Mila(米拉计算科学研究所);Dreamfold(Dreamfold公司);Duke University(杜克大学);McGill University(麦吉尔大学);Allen Institute for AI(艾伦人工智能研究所);Oxford University(牛津大学);Aithyra(Aithyra公司)
RAPiD: Reward-Guided Consistency Distillation of Diffusion Planners for Real-Time Autonomous Driving
School of Information Technology, Monash University, Malaysia(墨尔本大学信息科技学院);Faculty of Information Technology, Monash University, Australia(墨尔本大学信息技术学院);TCS Research, India(印度TCS研究)