Diffusion-based large language models (dLLMs) are gaining attention for their inherent capacity for parallel decoding, offering a compelling alternative to autoregressive LLMs. Among various decoding strategies, block-wise semi-autoregressive (semi-AR) approaches are widely adopted due to their support for KV caching and their favorable accuracy-speed trade-off. However, this paper identifies two fundamental limitations in the conventional semi-AR decoding approach that applies a fixed block size: i) late decoding overhead, where the unmasking of high-confidence tokens outside the current block is unnecessarily delayed, and ii) premature decoding error, where low-confidence tokens inside the current block are committed too early, leading to incorrect tokens. This paper presents the first systematic investigation challenging the fixed block size setting in semi-AR decoding. Through a statistical analysis of confidence dynamics during the denoising process, we identify a volatility band (VB) region during dLLM decoding, which encodes local semantic structure and can be used to guide adaptive block sizing. Leveraging these insights, we introduce AdaBlock-dLLM, a training-free, plug-and-play scheduler that adaptively aligns block boundaries with semantic steps by adjusting block size during runtime. Extensive experiments across diverse benchmarks show that AdaBlock-dLLM achieves up to 5.3% accuracy improvement under the same throughput budget. Beyond inference-time optimization, we hope our semantics-aware adaptive scheduling approach and confidence-based analysis will inspire future training strategies for dLLMs. Our code is available at https://github.com/lgxi24/AdaBlock-dLLM.
Streaming-dLLM: Accelerating Diffusion LLMs via Suffix Pruning and Dynamic Decoding
School of information(信息学院);Electronics, Beijing Institute of Technology, Beijing(电子学院,北京理工大学,北京);Department of Computer Science, City University of Hong Kong, Hong Kong(计算机科学系,香港城市大学,香港);School of Computer Science, Wuhan University, Wuhan(计算机科学学院,武汉大学,武汉);School of Economics(经济学院);Management, Communication University of China, Beijing(管理学院,中国传媒大学,北京);School of Science(科学学院);Engineering, The Chinese University of Hong Kong (Shenzhen), Shenzhen(工程学院,香港中文大学(深圳),深圳)
SimSD: Simple Speculative Decoding in Diffusion Language Models
University of California San Diego(加州大学圣地亚哥分校);University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);Google(谷歌);University of California Santa Barbara(加州大学圣芭芭拉分校)
d3LLM: Ultra-Fast Diffusion LLM using Pseudo-Trajectory Distillation
University of California, San Diego(加州大学圣地亚哥分校);School of Artificial Intelligence, Nanjing University(南京大学人工智能学院);National Key Laboratory for Novel Software Technology, Nanjing University(南京大学新型软件技术国家实验室);Shanghai Jiao Tong University(上海交通大学)
LLaDA2.0: Scaling Up Diffusion Language Models to 100B
Ant Group(蚂蚁集团);Renmin University of China(中国人民大学);Zhejiang University(浙江大学);Westlake University(西湖大学);HongKong University of Science and Technology(香港科学与技术大学)
DiffuGuard: How Intrinsic Safety is Lost and Found in Diffusion Large Language Models
Beijing University of Posts and Telecommunications(北京邮电大学);National University of Singapore(新加坡国立大学);Nanyang Technological University(南洋理工大学);Tsinghua University(清华大学);The Chinese University of Hong Kong(香港中文大学);Squirrel Ai Learning(松鼠AI);Peking University(北京大学)
SLIM-RL: Risk-Budgeted Random-Masking RL for Diffusion LLMs Without Trajectory Slicing
Technical University of Denmark(丹麦技术大学);MBZUAI Institute of Foundation Models(穆罕默德·本·扎耶德人工智能大学基础模型研究所);Iowa State University(爱荷华州立大学);Red Hat AI Innovation(红帽人工智能创新实验室);MIT–IBM Watson AI Lab(麻省理工学院-IBM沃森人工智能实验室)
Escaping Confidence Trap: Evolutionary Decoding for Mathematical Reasoning in Diffusion LLMs
Australian National University(澳大利亚国立大学);Nanyang Technological University(南洋理工大学);National University of Singapore(新加坡国立大学);Amazon(亚马逊);University of Technology Sydney(悉尼科技大学)
STaR-Quant: State-Time Consistent Post-Training Quantization for Diffusion Large Language Models
School of Artificial Intelligence, Beijing Normal University, Beijing, China(北京师范大学人工智能学院);Department of Computer Science, National University of Singapore, Singapore(新加坡国立大学计算机科学系);Centre for Frontier AI Research, Agency for Science, Technology and Research (A*STAR), Singapore(科技研究局前沿人工智能研究中心)
GroupKV: Hierarchical KV Cache Management for Long-Context Diffusion LLM Inference
Institute of Software, Chinese Academy of Sciences(中国科学院软件研究所);University of Chinese Academy of Sciences(中国科学院大学);Nanjing University of Science and Technology(南京理工大学)
Denoising Surface: Modeling and Predicting Inference Cost for Diffusion LLM Serving
Wuhan University(武汉大学);Shanghai Jiao Tong University(上海交通大学);The Hong Kong University of Science and Technology(香港科技大学);Damen Database Co., Ltd.(达梦数据库有限公司);Central China Normal University(华中师范大学)
Re-evaluating Confidence Remasking in Masked Diffusion Language Models
UvA-Bosch Delta Lab, University of Amsterdam(阿姆斯特丹大学UvA-Bosch Delta实验室);Bosch Center for AI(博世人工智能中心);University of Basel(巴塞尔大学);Johns Hopkins University(约翰霍普金斯大学)