Large Language Models (LLMs) demonstrate their reasoning ability through chain-of-thought (CoT) generation. However, LLM's autoregressive decoding may limit the ability to revisit and refine earlier tokens in a holistic manner, which can also lead to inefficient exploration for diverse solutions. In this paper, we propose LaDiR} (Latent Diffusion Reasoner), a novel reasoning framework that unifies the expressiveness of continuous latent representation with the iterative refinement capabilities of latent diffusion models for an existing LLM. We first construct a structured latent reasoning space using a Variational Autoencoder (VAE) that encodes text reasoning steps into blocks of thought tokens, preserving semantic information and interpretability while offering compact but expressive representations. Subsequently, we utilize a latent diffusion model that learns to denoise a block of latent thought tokens with a blockwise bidirectional attention mask, enabling longer horizon and iterative refinement with adaptive test-time compute. This design, combined with explicit diversity guidance during diffusion inference, enables the generation of multiple diverse reasoning trajectories that explore distinct regions of the latent space, rather than producing repetitive solutions as often occurs in standard autoregressive sampling. We conduct evaluations on a suite of mathematical reasoning, code generation and puzzle planning benchmarks. Empirical results show that LaDiR consistently improves accuracy, diversity, and interpretability over existing autoregressive, diffusion-based, and latent reasoning methods, revealing a new paradigm for text reasoning with latent diffusion.
Parallel Test-Time Scaling for Latent Reasoning Models
Hong Kong Polytechnic University（香港理工大学）；Shandong Jianzhu University（山东建筑大学）；University of Science and Technology of China（中国科学技术大学）；Harbin Institute of Technology (Shenzhen)（哈尔滨工业大学（深圳））
Seek in the Dark: Reasoning via Test-Time Instance-Level Policy Gradient in Latent Space
Institute for Artificial Intelligence, Peking University（北京大学人工智能研究院）；NLCo Lab, Beijing Institute for General Artificial Intelligence（北京通用人工智能研究院NLCo实验室）；Department of Automation, Tsinghua University（清华大学自动化系）；Shanghai Jiao Tong University（上海交通大学）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；University of California, Los Angeles（加州大学洛杉矶分校）
Beyond Dense States: Sparse Transcoders as Causally Testable Operators for LLM Latent Reasoning
College of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics（南京航空航天大学计算机科学与技术学院）；MIIT Key Laboratory of Pattern Analysis and Machine Intelligence（信息产业部模式分析与机器智能重点实验室）；Institute for AI, Tsinghua University, Beijing, China（清华大学人工智能研究院）
Instituto Superior Técnico, Universidade de Lisboa（里斯本大学理工学院）；Instituto de Telecomunicações（电信研究所）；TransPerfect（TransPerfect公司）；Carnegie Mellon University（卡内基梅隆大学）
What Makes Effective Supervision in Latent Chain-of-Thought: An Information-Theoretic Analysis
Ningbo Institute of Digital Twin, Eastern Institute of Technology（宁波数字孪生研究院，东方理工大学）；Department of Computing, The Hong Kong Polytechnic University（香港理工大学计算学系）
Unlocking the Working Memory of Large Language Models for Latent Reasoning
ELLIS Unit Linz and LIT AI Lab, Institute for Machine Learning, Johannes Kepler University Linz, Austria（林茨ELLIS单元和LIT AI实验室，机器学习研究所，约翰·凯撒大学林茨，奥地利）；NXAI GmbH, Linz, Austria（NXAI公司，林茨，奥地利）
