Latent reasoning has been recently proposed as a reasoning paradigm and performs multi-step reasoning through generating steps in the latent space instead of the textual space. This paradigm enables reasoning beyond discrete language tokens by performing multi-step computation in continuous latent spaces. Although there have been numerous studies focusing on improving the performance of latent reasoning, its internal mechanisms remain not fully investigated. In this work, we conduct a comprehensive analysis of latent reasoning methods to better understand the role and behavior of latent representation in the process. We identify two key issues across latent reasoning methods with different levels of supervision. First, we observe pervasive shortcut behavior, where they achieve high accuracy without relying on latent reasoning. Second, we examine the hypothesis that latent reasoning supports BFS-like exploration in latent space, and find that while latent representations can encode multiple possibilities, the reasoning process does not faithfully implement structured search, but instead exhibits implicit pruning and compression. Finally, our findings reveal a trade-off associated with supervision strength: stronger supervision mitigates shortcut behavior but restricts the ability of latent representations to maintain diverse hypotheses, whereas weaker supervision allows richer latent representations at the cost of increased shortcut behavior.
Training Large Language Models for Reasoning through Reverse Curriculum Reinforcement Learning
School of Computer Science, Fudan University(复旦大学计算机科学学院);ByteDance Research(字节跳动研究部);Institute of Modern Languages and Linguistics, Fudan University(复旦大学现代语言与语言学研究所)
Parallel Test-Time Scaling for Latent Reasoning Models
Hong Kong Polytechnic University(香港理工大学);Shandong Jianzhu University(山东建筑大学);University of Science and Technology of China(中国科学技术大学);Harbin Institute of Technology (Shenzhen)(哈尔滨工业大学(深圳))
Seek in the Dark: Reasoning via Test-Time Instance-Level Policy Gradient in Latent Space
Institute for Artificial Intelligence, Peking University(北京大学人工智能研究院);NLCo Lab, Beijing Institute for General Artificial Intelligence(北京通用人工智能研究院NLCo实验室);Department of Automation, Tsinghua University(清华大学自动化系);Shanghai Jiao Tong University(上海交通大学);Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);University of California, Los Angeles(加州大学洛杉矶分校)
Beyond Dense States: Sparse Transcoders as Causally Testable Operators for LLM Latent Reasoning
College of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics(南京航空航天大学计算机科学与技术学院);MIIT Key Laboratory of Pattern Analysis and Machine Intelligence(信息产业部模式分析与机器智能重点实验室);Institute for AI, Tsinghua University, Beijing, China(清华大学人工智能研究院)
KnowCoder-A1: Incentivizing Agentic Reasoning Capability with Outcome Supervision for KBQA
Institute of Computing Technology, Chinese Academy of Sciences(中国科学院计算技术研究所);State Key Laboratory of AI Safety(人工智能安全国家重点实验室);School of Computer Science, University of Chinese Academy of Sciences(中国科学院大学计算机科学学院)
KG-TRACES: Enhancing Large Language Models with Knowledge Graph-constrained Trajectory Reasoning and Attribution Supervision
Zhejiang University(浙江大学);Shanghai Artificial Intelligence Laboratory(上海人工智能实验室);University of Science and Technology of China(中国科学技术大学);Fudan University(复旦大学)
Instituto Superior Técnico, Universidade de Lisboa(里斯本大学理工学院);Instituto de Telecomunicações(电信研究所);TransPerfect(TransPerfect公司);Carnegie Mellon University(卡内基梅隆大学)
Evo-PI: Aligning Medical Reasoning via Evolving Principle-Guided Supervision
School of Computer Science, University of Auckland(奥克兰大学计算机科学学院);Beijing University of Posts and Telecommunications(北京邮电大学);Renyixun Health Technology Co., Ltd.(仁医讯健康科技有限公司);National Yang Ming Chiao Tung University(国立阳明交通大学);Harbin Institute of Technology (Shenzhen)(哈尔滨工业大学(深圳))
VITAL: Visual-Semantic Dual Supervision for Enhanced and Interpretable Latent Reasoning in Medical MLLMs
Zhejiang University(浙江大学);Shanghai AI Laboratory(上海人工智能实验室);Tencent(腾讯);Ningbo Global Innovation Center, Zhejiang University(宁波全球创新中心,浙江大学);Zhejiang Key Laboratory of Digital-Intelligence Service Technology(浙江省数字智能服务技术重点实验室)
ReasonCLIP-58M: Visually Grounded Commonsense Reasoning Supervision for CLIP
Khalifa University(卡利法大学);University of Western Australia(西澳大学);The Chinese University of Hong Kong(香港中文大学);University of Melbourne(墨尔本大学);University of Central Florida(佛罗里达中央大学)
Correct Answers from Sound Reasoning: Verifiable Process Supervision for Language Models
KAIST AI(KAIST人工智能研究所);University of Texas, Austin(德克萨斯大学奥斯汀分校);Rice University(里士满大学);Princeton University(普林斯顿大学);University of Washington(华盛顿大学);Sentient Labs(Sentient实验室)
Reasoning in the Dark: Interleaved Vision-Text Reasoning in Latent Space
The Hong Kong Polytechnic University(香港理工大学);Singapore Management University(新加坡管理学院);Shandong University(山东大学);Harbin Institute of Technology (Shenzhen)(哈尔滨工业大学(深圳))
MoLE: Mixture of Latent Experts for Complementary Visual Reasoning
Tongji University(同济大学);Hong Kong Polytechnic University(香港理工大学);Alibaba Group(阿里巴巴集团);University of Electronic Science and Technology of China(电子科技大学)