We study how to extend chain-of-thought (CoT) beyond language to better handle multimodal reasoning. While CoT helps LLMs and VLMs articulate intermediate steps, its text-only form often fails on vision-intensive problems where key intermediate states are inherently visual. We introduce modal-mixed CoT, which interleaves textual tokens with compact visual sketches represented as latent embeddings. To bridge the modality gap without eroding the original knowledge and capability of the VLM, we use the VLM itself as an encoder and train the language backbone to reconstruct its own intermediate vision embeddings, to guarantee the semantic alignment of the visual latent space. We further attach a diffusion-based latent decoder, invoked by a special control token and conditioned on hidden states from the VLM. In this way, the diffusion head carries fine-grained perceptual details while the VLM specifies high-level intent, which cleanly disentangles roles and reduces the optimization pressure of the VLM. Training proceeds in two stages: supervised fine-tuning on traces that interleave text and latents with a joint next-token and latent-reconstruction objective, followed by reinforcement learning that teaches when to switch modalities and how to compose long reasoning chains. Extensive experiments across 11 diverse multimodal reasoning tasks, demonstrate that our method yields better performance than language-only and other CoT methods. Our code will be publicly released.
Separate First, Fuse Later: Mitigating Cross-Modal Interference in Audio-Visual LLMs Reasoning with Modality-Specific Chain-of-Thought
Tianjin Key Laboratory of Cognitive Computing（天津认知计算实验室）；Tianjin University（天津大学）；Huiyan Technology Company, Ltd.（慧颜科技有限公司）；Chinese Academy of Sciences（中国科学院）；Tencent（腾讯）
CoMT: A Novel Benchmark for Chain of Multi-modal Thought on Large Vision-Language Models
School of Computer Science and Engineering, Central South University（中南大学计算机科学与工程学院）；Key Laboratory of Data Intelligence and Advanced Computing in Provincial Universities, Soochow University（苏州大学省级高校数据智能与先进计算重点实验室）；Research Center for SCIR, Harbin Institute of Technology（哈尔滨工业大学SCIR研究中心）；National University of Singapore（新加坡国立大学）
M$^3$CoT: A Novel Benchmark for Multi-Domain Multi-step Multi-modal Chain-of-Thought
Research Center for Social Computing and Information Retrieval（社会计算与信息检索研究中心）；Harbin Institute of Technology（哈尔滨工业大学）；School of Computer Science and Engineering, Central South University（中南大学计算机科学与工程学院）；Shanghai AI Laboratory（上海人工智能实验室）
A*-Thought-V2: Efficient Latent Reasoning via Geometric Dynamics of LLM
Beijing University of Posts and Telecommunications（北京邮电大学）；The Hong Kong Polytechnic University（香港理工大学）；Tsinghua University（清华大学）；JIUTIAN Research（中移九天）；OpenBMB
School of Computer Science and Technology, Soochow University（苏州大学计算机科学与技术学院）；Department of Computer Science, The Hong Kong Polytechnic University（香港理工大学计算机科学系）
GraphMed-LT: Patient-Specific Graph Memory with Latent Clinical Thought Refinement for Multi-Turn Medical Conversations
School of Computing Science, University of Glasgow, UK（格拉斯哥大学计算机科学学院）；School of Natural and Computing Science, University of Aberdeen, UK（阿伯丁大学自然与计算科学学院）
Think Consistently, Reason Efficiently: Energy-Based Calibration for Implicit Chain-of-Thought
University of Oxford（牛津大学）；Tsinghua University（清华大学）；Tencent（腾讯）；Southern University of Science and Technology（南方科技大学）；National University of Singapore（新加坡国立大学）
Balancing Efficiency and Efficacy: Training-Free Attention-Guided Switching Between Explicit and Latent Thoughts for MLLMs
Harbin Institute of Technology, Shenzhen（哈尔滨工业大学（深圳））；Tsinghua Shenzhen International Graduate School, Tsinghua University（清华大学深圳国际研究生院）；Peng Cheng Laboratory（鹏城实验室）；Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）
Soft-Prompting with Graph-of-Thought for Multi-modal Representation Learning
School of Computer Science, Wuhan University（武汉大学计算机学院）；National Engineering Research Center for Multimedia Software（国家多媒体软件工程技术研究中心）；School of Electronic Information Engineering, Henan Polytechnic Institute（河南工业职业技术学院电子信息工程学院）；JD Explore Academy（京东探索研究院）
LaViT: Aligning Latent Visual Thoughts for Multi-modal Reasoning
City University of Hong Kong（香港城市大学）；University of Science and Technology of China（中国科学技术大学）；Utrecht University（乌特勒支大学）；University of Electronic Science and Technology of China（电子科技大学）
University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；University at Buffalo（布法罗大学）；Amazon（亚马逊）
Contrastive loss has been increasingly used in learning representations from multiple modalities. In the limit, the nature of the contrastive loss encourages modalities to exactly match each other in the latent space. Yet it remains an open question how the modality alignment affects the downstream task performance. In this paper, based on an information-theoretic argument, we first prove that exact modality alignment is sub-optimal in general for downstream prediction tasks. Hence we advocate that the key of better performance lies in meaningful latent modality structures instead of perfect m…
CritiqueDriveVLM: From Verifier-Guided Reinforcement Learning to Latent Thought Distillation for Autonomous Driving
State Key Laboratory of Networking and Switching Technology（网络与交换技术国家重点实验室）；School of Digital Media & Design Arts（数字媒体与设计艺术学院）；Beijing University of Posts and Telecommunications（北京邮电大学）
Capabilities and Fundamental Limits of Latent Chain-of-Thought
Department of Mathematics and Statistics, Xi'an Jiaotong University（西安交通大学数学与统计学系）；Department of Artificial Intelligence, Gaoling School of Artificial Intelligence, Renmin University of China（中国人民大学人工智能学院）
Do Latent-CoT Models Think Step-by-Step? A Mechanistic Study on Sequential Reasoning Tasks
Institute for Computational and Mathematical Engineering, Stanford University, Stanford, USA（计算与数学工程研究所，斯坦福大学）；MOE Key Lab of Computational Linguistics, Peking University, Beijing, China（教育部计算语言学重点实验室，北京大学）；Beijing Academy of Artificial Intelligence, Beijing, China（北京人工智能研究院）
ASLSL: Adaptive shared latent structure learning with incomplete multi-modal physiological data for multi-dimensional emotional feature selection
School of Information Science and Technology, Beijing University of Technology, Beijing 100124, China（信息科学与技术学院，北京理工大学，北京）；School of Artificial Intelligence, Anhui University, Beijing 100124, China（人工智能学院，安徽大学，北京）
