Lookahead Path Likelihood Optimization for Diffusion LLMs
Institute for Artificial Intelligence, Peking University（人工智能研究院，北京大学）；School of Computing, National University of Singapore（计算学院，新加坡国立大学）；School of Intelligence Science and Technology, Peking University（智能科学与技术学院，北京大学）
Diffusion Large Language Models (dLLMs) support arbitrary-order generation, yet their inference performance critically depends on the unmasking order. Existing strategies rely on heuristics that greedily optimize local confidence, offering limited guidance for identifying unmasking paths that are globally consistent and accurate. To bridge this gap, we introduce path log-likelihood (Path LL), a trajectory-conditioned objective that strongly correlates with downstream accuracy and enables principled selection of unmasking paths. To optimize Path LL at inference time, we propose POKE, an efficient value estimator that predicts the expected future Path LL of a partial decoding trajectory. We then integrate this lookahead signal into POKE-SMC, a Sequential Monte Carlo-based search framework for dynamically identifying optimal unmasking paths. Extensive experiments across 6 reasoning tasks show that POKE-SMC consistently improves accuracy, achieving 2%--3% average gains over strong decoding-time scaling baselines at comparable inference overhead on LLaDA models and advancing the accuracy--compute Pareto frontier.
LookAhead Tuning: Safer Language Models via Partial Answer Previews
Zhejiang University（浙江大学）；Zhejiang University - Ant Group Joint Laboratory of Knowledge Graph（浙江大学-蚂蚁集团知识图谱联合实验室）；Ant Group（蚂蚁集团）；National University of Singapore（新加坡国立大学）
TeleRAG: Efficient Retrieval-Augmented Generation Inference with Lookahead Retrieval
Paul G. Allen School of Computer Science and Engineering, University of Washington, Seattle, WA, USA（华盛顿大学保罗·G·艾伦计算机科学与工程学院，西雅图，华盛顿州，美国）；Harvard John A. Paulson School of Engineering and Applied Sciences, Harvard University, Cambridge, MA, USA（哈佛大学约翰·A·保罗森工程与应用科学学院，剑桥，马萨诸塞州，美国）；Shanghai Jiao Tong University, Shanghai, China（上海交通大学，上海，中国）
Principled RL for Diffusion LLMs Emerges from a Sequence-Level Perspective
Gaoling School of Artificial Intelligence, Renmin University of China（中国人民大学人工智能学院）；Beijing Key Laboratory of Research on Large Models and Intelligent Governance（北京大模型与智能治理研究重点实验室）；Engineering Research Center of Next-Generation Intelligent Search and Recommendation, MOE（下一代智能搜索与推荐工程技术研究中心）；Stanford University（斯坦福大学）；Lambda, Inc（Lambda公司）；Tsinghua University（清华大学）
