While Multimodal Large Language Models (MLLMs) have achieved impressive performance on semantic tasks, their spatial intelligence--crucial for robust and grounded AI systems--remains underdeveloped. Existing benchmarks fall short of diagnosing this limitation: they either focus on overly simplified qualitative reasoning or rely on domain-specific indoor data, constrained by the lack of outdoor datasets with verifiable metric ground truth. To bridge this gap, we introduce a large-scale benchmark built from pedestrian-perspective videos captured with synchronized stereo cameras, LiDAR, and IMU/GPS sensors. This dataset provides metrically precise 3D information, enabling the automatic generation of spatial reasoning questions that span a hierarchical spectrum--from qualitative relational reasoning to quantitative metric and kinematic understanding. Evaluations reveal that the performance gains observed in structured indoor benchmarks vanish in open-world settings. Further analysis using synthetic abnormal scenes and blinding tests confirms that current MLLMs depend heavily on linguistic priors instead of grounded visual reasoning. Our benchmark thus provides a principled platform for diagnosing these limitations and advancing physically grounded spatial intelligence.
RedundancyLens: Revealing and Exploiting Visual Token Processing Redundancy for Efficient Decoder-Only MLLMs
South China University of Technology(华南理工大学);Intsig Information Co., Ltd.(Intsig信息有限公司);INTSIG-SCUT Joint Lab on Document Analysis and Recognition(INTSIG-SCUT文档分析与识别联合实验室)
Beyond Visual Evidence: Revealing and Mitigating Relational Privacy Leakage in Document MLLMs
Faculty of Engineering, Shenzhen MSU-BIT University(深圳北理莫斯科大学工程学院);Faculty of CMC, Shenzhen MSU-BIT University(深圳北理莫斯科大学计算机、数学与力学学院);Department of CDS, Indian Institute of Science(印度科学学院计算机与数据科学系)
RADAR: Revealing Asymmetric Development of Abilities in MLLM Pre-training
Shenzhen Campus of Sun Yat-sen University(中山大学深圳校区);Peng Cheng Laboratory(鹏城实验室);Guangdong Key Laboratory of Big Data Analysis and Processing(广东大数据分析与处理重点实验室);Tsinghua Shenzhen International Graduate School(清华大学深圳国际 Graduate School);Tsinghua University(清华大学);Shanghai Jiao Tong University(上海交通大学);Yinwang Intelligent Technology Co., Ltd.(亿纬智能科技有限公司);Huawei’s 2012 Lab(华为2012实验室)
Reinforcing Egocentric Spatial Perception in Multimodal Large Language Models via Ego Scene Augmentation
The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));Guangxi Zhuang Autonomous Region Information Center(广西壮族自治区信息中心);The Hong Kong University of Science and Technology(香港科技大学)
CardioLens: Revealing the Clinical Reality Gap of MLLMs via Multi-Sequence Cardiac MRI Evaluations
Beijing Academy of Artificial Intelligence(北京人工智能研究院);Beijing Anzhen Hospital(北京安贞医院);Beihang University(北航);King Abdullah University of Science and Technology(国王 Abdullah 科学与技术大学)
Can MLLMs Reason Beyond Language? VisReason: A Comprehensive Benchmark for Vision-Centric Reasoning
Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院)
ViSTR-Bench: Can MLLMs Reason from Continuous Visual Cues in Dynamic Scenes?
School of Artificial Intelligence, Beihang University(北京航空航天大学人工智能学院);Zhongguancun Academy(中关村科学城);School of Engineering, Westlake University(西湖大学工学院);School of Transportation Science and Engineering, Beihang University(北京航空航天大学交通科学与工程学院)
Customizing Visual Emotion Evaluation for MLLMs: An Open-vocabulary, Multifaceted, and Scalable Approach
Institute of Information Engineering, Chinese Academy of Sciences(中国科学院信息工程研究所);VCIP & TMCC & DISSec, College of Computer Science, Nankai University(南开大学计算机学院);Department of Psychological and Cognitive Sciences, Tsinghua University(清华大学心理学与认知科学系);University of Chinese Academy of Sciences(中国科学院大学)
RoadBench: Benchmarking MLLMs on Fine-Grained Spatial Understanding and Reasoning under Urban Road Scenarios
Department of Electronic Engineering, BNRist, Tsinghua University(清华大学电子工程系、北京信息科学与技术国家研究中心);Zhongguancun Academy(中关村学院);Amap, Alibaba Group(阿里巴巴集团高德地图)
GroundingME: Exposing the Visual Grounding Gap in MLLMs through Multi-Dimensional Evaluation
State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University(多媒体信息处理国家重点实验室,计算机学院,北京大学);LLM-Core Xiaomi(小米LLM-Core);The University of Hong Kong(香港大学);Renmin University of China(中国人民大学)
A Benchmark Dataset for MLLM-Generated Image Detection: GPT Image2 & Nano Banana2
College of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics(南京航空航天大学计算机科学与技术学院);College of Artificial Intelligence, Nanjing University of Aeronautics and Astronautics(南京航空航天大学人工智能学院);School of Computer Science and Technology, Harbin Institute of Technology(哈尔滨工业大学计算机科学与技术学院)
GI-Bench: A Panoramic Benchmark Revealing the Knowledge-Experience Dissociation of Multimodal Large Language Models in Gastrointestinal Endoscopy Against Clinical Standards
Zhongshan Hospital, Fudan University(复旦大学附属中山医院);School of Basic Medical Sciences, Fudan University(复旦大学基础医学院);Microsoft Research Asia(微软亚洲研究院);Data Science Institute, Imperial College London(帝国理工学院数据科学研究所)
NavBench: Probing Multimodal Large Language Models for Embodied Navigation
The University of Adelaide(阿德莱德大学);The University of Queensland(昆士兰大学);CSIRO Data61(澳大利亚联邦科学与工业研究组织数据61研究中心);Mohamed bin Zayed University of Artificial Intelligence(穆罕默德·本·扎耶德人工智能大学);Tongji University(同济大学)
CoVFT: Context-aware Visual Fine-tuning for Multimodal Large Language Models
State Key Laboratory of Complex and Critical Software Environment, Beihang University(北京航空航天大学复杂关键软件环境国家重点实验室);School of Computer Science and Engineering, Beihang University(北京航空航天大学计算机科学与工程学院)
ViC-Bench: Benchmarking Visual-Interleaved Chain-of-Thought Capability in MLLMs with Free-Style Intermediate State Representations
School of Computer Science and Technology, Xi’an Jiaotong University(西安交通大学计算机科学与技术学院);Meituan Inc.(美团公司);Institute of Advanced Technology, University of Science and Technology of China(中国科学技术大学先进技术研究院);School of Computer Science and Technology, East China Normal University(华东师范大学计算机科学与技术学院);Research Center for Space Computing System, Zhejiang Lab(浙江实验室空间计算系统研究中心);College of Electronic and Information Engineering, Tongji University(同济大学电子与信息工程学院);School of Robotics and Automation, Nanjing University(南京大学机器人与自动化学院);College of Intelligent Robotics and Advanced Manufacturing, Fudan University(复旦大学智能机器人与先进制造学院)
LLaVA-UHD v3: Progressive Visual Compression for Efficient Native-Resolution Encoding in MLLMs
Tsinghua University(清华大学);University of Chinese Academy of Sciences(中国科学院大学);Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);Aerospace Information Research Institute, Chinese Academy of Sciences(中国科学院航天信息研究所);School of Advanced Interdisciplinary Sciences, University of Chinese Academy of Sciences(中国科学院大学先进交叉学科学院)
AesTest: Measuring Aesthetic Intelligence from Perception to Production
University of International Business and Economics(国际商务经济大学);University of Science and Technology of China(中国科学技术大学);Huawei Technologies Co., Ltd(华为技术有限公司);Beijing Electronic Science and Technology Institute(北京电子科技研究所);Beijing Institute for General Artificial Intelligence(北京通用人工智能研究院)
OmniBrainBench: A Comprehensive Multimodal Benchmark for Brain Imaging Analysis Across Multi-stage Clinical Tasks
Chinese University of Hong Kong(香港中文大学);Sun Yat-sen Memorial Hospital, Sun Yat-sen University(中山大学孙逸仙纪念医院);Southern Medical University(南方医科大学);Zhongshan Hospital, Fudan University(复旦大学中山医院);Department of Neurosurgery, Prince of Wales Hospital(威尔士王室医院神经外科部)
Warehouse Spatial Question Answering with LLM Agent
Information Processing Lab, University of Washington, USA(华盛顿大学信息处理实验室);Electronics and Telecommunications Research Institute, South Korea(韩国电子电信研究院);National Center for High-performance Computing, Taiwan(台湾高性能计算国家研究中心)
本文提出一种数据高效的 LLM agent 系统,通过集成多种工具进行空间推理与 API 交互,解决了复杂室内仓库场景下的空间问答任务,并在目标检索、计数和距离估计上实现了高准确率与高效率。
Can MLLMs Read Students' Minds? Unpacking Multimodal Error Analysis in Handwritten Math
Lehigh University(里海大学);Squirrel Ai Learning(松鼠AI学习);Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);Michigan State University(密歇根州立大学)
Revisiting Visual Understanding in Multimodal Reasoning through a Lens of Image Perturbation
School of Computer Science, Shanghai Jiao Tong University(上海交通大学计算机科学学院);Shanghai Innovation Institute(上海创新研究院);Zhongguancun Academy(中关村学院);Tencent(腾讯);State Key Laboratory of General Artificial Intelligence, BIGAI(通用人工智能国家重点实验室,BIGAI);Lehigh University(莱斯大学)
Visual Question Decomposition on Multimodal Large Language Models
Munich Center for Machine Learning(慕尼黑机器学习中心);Technical University of Munich(慕尼黑工业大学);Amazon Web Services(亚马逊云计算服务);LMU Munich(慕尼黑大学);MBZUAI(穆罕默德·本·扎耶德人工智能大学);University of Oxford(牛津大学)
Evaluating and Advancing Multimodal Large Language Models in Perception Ability Lens
AIML, University of Adelaide(AIML,阿德莱德大学);Monash University(墨尔本大学);Australian National University(澳大利亚国立大学);Tiktok, Australia(Tiktok,澳大利亚);Zhejiang University(浙江大学)
SpaceDG: Benchmarking Spatial Intelligence under Visual Degradation
Shanghai Jiao Tong University(上海交通大学);Shanghai Artificial Intelligence Laboratory(上海人工智能实验室);University of Electronic Science and Technology of China(电子科技大学);Chongqing University(重庆大学);The University of Tokyo(东京大学);Beihang University(北航);Northwestern Polytechnical University(西北工业大学)