Large Vision Language Models (LVLMs) excel at semantic understanding but struggle with fine grained spatial grounding, as the model must implicitly infer complex geometry without ever producing a spatial interpretation. We present Perceptio, a perception enhanced LVLM with 2D and 3D spatial reasoning abilities, enabled via explicit semantic segmentation tokens and depth tokens generated directly within the autoregressive sequence. Concretely, we (i) distill a VQVAE depth codebook from a strong monocular teacher to tokenize dense depth into compact sequences, and (ii) integrate SAM2 based semantic segmentation tokens and VQ-VAE depth tokens inside the LLM so the model first emits spatial tokens and then answers. To stabilize depth token generation, we introduce novel composite depth-token objectives (marker, token, and count losses) and a soft-merging technique for differentiable reconstruction. We adopt a multi-task co-training strategy across diverse datasets, letting the model learn perception tokens to tackle multiple downstream tasks. Building on InternVL, Perceptio achieves state-of-the-art performance across benchmarks: improving referring expression segmentation by +0.8/+1.4/+1.1 cIoU on RefCOCO/+/g HardBLINK spatial understanding accuracy by 10.3%, and MMBench accuracy by 1.0%, demonstrating that explicit spatial chain-of-thought materially strengthens spatial grounding in LVLMs.
Perception-R1: Advancing Multimodal Reasoning Capabilities of MLLMs via Visual Perception Reward
University of Science and Technology of China(中国科学技术大学);State Key Laboratory of Cognitive Intelligence(认知智能国家重点实验室);iFLYTEK AI Research(iFLYTEK人工智能研究院);Hong Kong University of Science and Technology(香港科学与技术大学)
Perception-R1: Pioneering Perception Policy with Reinforcement Learning
Huazhong University of Science and Technology(华中科技大学);Beijing University of Posts and Telecommunications(北京邮电大学);StepFun(阶跃星辰);Johns Hopkins University(约翰斯·霍普金斯大学)
Unleashing Perception-Time Scaling to Multimodal Reasoning Models
Gaoling School of Artificial Intelligence, Renmin University of China(中国人民大学北京校区人工智能学院);Beijing Key Laboratory of Research on Large Models and Intelligent Governance(北京大模型与智能治理重点实验室);ByteDance(字节跳动);University of California, San Diego(加州大学圣地亚哥分校);Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所)
Dynamic V2X Autonomous Perception from Road-to-Vehicle Vision
Suzhou University of Science and Technology(苏州科技大学);CRIPAC(中国科学院自动化研究所模式识别国家重点实验室);CASIA(中国科学院自动化研究所);Suzhou Institute of Trade Commerce(苏州经贸职业技术学院);Tianjin University(天津大学)
APPO: Attention-guided Perception Policy Optimization for Video Reasoning
Gaoling School of Artificial Intelligence, Renmin University of China, Beijing(中国人民大学北京校区人工智能学院);AI Technology Center, Online Video Business Unit, Tencent PCG(腾讯PCG人工智能技术中心)
Vision-Driven 2D Supervised Fine-Tuning Framework for Bird's Eye View Perception
School of Vehicle and Mobility, Tsinghua University(清华大学车辆与 Mobility学院);State Key Laboratory of Intelligent Green Vehicle and Mobility, Tsinghua University(清华大学智能绿色车辆与交通全国重点实验室)