Spatial reasoning has emerged as a critical capability for Multimodal Large Language Models (MLLMs), drawing increasing attention and rapid advancement. However, existing benchmarks primarily focus on single-step perception-to-judgment tasks, leaving scenarios requiring complex visual-spatial logical chains significantly underexplored. To bridge this gap, we introduce Video-MSR, the first benchmark specifically designed to evaluate Multi-hop Spatial Reasoning (MSR) in dynamic video scenarios. Video-MSR systematically probes MSR capabilities through four distinct tasks: Constrained Localization, Chain-based Reference Retrieval, Route Planning, and Counterfactual Physical Deduction. Our benchmark comprises 3,052 high-quality video instances with 4,993 question-answer pairs, constructed via a scalable, visually-grounded pipeline combining advanced model generation with rigorous human verification. Through a comprehensive evaluation of 20 state-of-the-art MLLMs, we uncover significant limitations, revealing that while models demonstrate proficiency in surface-level perception, they exhibit distinct performance drops in MSR tasks, frequently suffering from spatial disorientation and hallucination during multi-step deductions. To mitigate these shortcomings and empower models with stronger MSR capabilities, we further curate MSR-9K, a specialized instruction-tuning dataset, and fine-tune Qwen-VL, achieving a +7.82% absolute improvement on Video-MSR. Our results underscore the efficacy of multi-hop spatial instruction data and establish Video-MSR as a vital foundation for future research. The code and data will be available at https://github.com/ruiz-nju/Video-MSR.
ABMAMBA: Multimodal Large Language Model with Aligned Hierarchical Bidirectional Scan for Efficient Video Captioning
Keio University(庆应义塾大学);National Institute of Informatics(国立信息学研究所);National Institute of Informatics Research and Development Center for Large Language Models(国立信息学研究所大型语言模型研发中心)
M$^3$-VQA: A Benchmark for Multimodal, Multi-Entity, Multi-Hop Visual Question Answering
Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院)
Customizing Visual Emotion Evaluation for MLLMs: An Open-vocabulary, Multifaceted, and Scalable Approach
Institute of Information Engineering, Chinese Academy of Sciences(中国科学院信息工程研究所);VCIP & TMCC & DISSec, College of Computer Science, Nankai University(南开大学计算机学院);Department of Psychological and Cognitive Sciences, Tsinghua University(清华大学心理学与认知科学系);University of Chinese Academy of Sciences(中国科学院大学)
LM4LV: A Frozen Large Language Model for Low-level Vision Tasks
Shanghai Jiao Tong University(上海交通大学);Shanghai AI Laboratory(上海人工智能实验室);Nanjing University(南京大学);Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences(中国科学院深圳先进技术研究院)
University of Chinese Academy of Sciences(中国科学院大学);New Laboratory of Pattern Recognition (NLPR), CASIA(中国科学院自动化研究所模式识别国家重点实验室);Harbin Institute of Technology(哈尔滨工业大学);Shandong University(山东大学)
Q-Bench+: A Benchmark for Multi-modal Foundation Models on Low-level Vision from Single Images to Pairs
Institute of Image Communication and Network Engineering, Shanghai Jiao Tong University(上海交通大学图像通信与网络工程研究所);Nanyang Technological University(南洋理工大学);School of Computer Science and Engineering, Nanyang Technological University(南洋理工大学计算机科学与工程学院)
The Instinctive Bias: Spurious Images lead to Illusion in MLLMs
The Hong Kong Polytechnic University(香港理工大学);The Hong Kong University of Science and Technology(香港科技大学);University of Illinois at Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);NVIDIA(英伟达)
Shenzhen International Graduate School, Tsinghua University(清华大学深圳国际研究生院);Baidu Inc.(百度公司);The University of Sydney(悉尼大学);Amazon(亚马逊公司);The Chinese University of Hong Kong(香港中文大学)
HumanPCR: Probing MLLM Capabilities in Diverse Human-Centric Scenes
Key Laboratory of Intelligent Information Processing of Chinese Academy of Sciences (CAS), Institute of Computing Technology, CAS, China(中国科学院智能信息处理重点实验室(中国科学院)、计算技术研究所、中国)
Beyond Filtering: Adaptive Image-Text Quality Enhancement for MLLM Pretraining
University of Chinese Academy of Sciences (UCAS)(中国科学院大学);Baichuan Inc.(百川智能公司);New Laboratory of Pattern Recognition (NLPR)(新模式识别实验室);Institute of Automation, Chinese Academy of Sciences (CASIA)(中国科学院自动化研究所);Gaoling School of Artificial Intelligence, Renmin University of China(中国人民大学高瓴人工智能学院)
MMReason: An Open-Ended Multi-Modal Multi-Step Reasoning Benchmark for MLLMs Toward AGI
Nanyang Technological University(南洋理工大学);Tsinghua University(清华大学);Baidu Inc.(百度公司);University of California(加州大学);University of Science and Technology of China(中国科学技术大学)
DocHop: Benchmarking Out-of-domain Multi-hop Reasoning in Information-Dense Documents
University of Wisconsin-Madison(威斯康星大学麦迪逊分校);Massachusetts Institute of Technology(麻省理工学院);Stanford University(斯坦福大学);Kookmin University(国民大学);MIT-IBM Watson AI Lab, IBM Research(麻省理工学院-IBM沃森人工智能实验室,IBM研究院)
UrbanGround: From Local Perception to Spatial Agency in a Real-Scale City
Shanghai Jiao Tong University(上海交通大学);National University of Singapore(新加坡国立大学);Meituan(美团);The Chinese University of Hong Kong(香港中文大学);Shanghai University(上海大学);University of Oxford(牛津大学)
Quantifying and Mitigating Unimodal Biases in Multimodal Large Language Models: A Causal Perspective
Shanghai AI Laboratory(上海人工智能实验室);State Key Laboratory of General Artificial Intelligence, Peking University(北京大学通用人工智能国家重点实验室);School of Intelligence Science and Technology, Peking University(北京大学智能科学与技术学院);School of Computer Science, Fudan University(复旦大学计算机科学技术学院);Shanghai Artificial Intelligence Laboratory(上海人工智能实验室)
Fudan University(复旦大学);OpenGVLab(开放视觉实验室);Shanghai AI Laboratory(上海人工智能实验室);Tsinghua University(清华大学);The Chinese University of Hong Kong(香港中文大学);The University of Hong Kong(香港大学);Nanjing University(南京大学);SenseTime Research(商汤科技研究院)
MME-Emotion: A Holistic Evaluation Benchmark for Emotional Intelligence in Multimodal Large Language Models
The Chinese University of Hong Kong(香港中文大学);Tongyi Lab(通义实验室);SZTU(深圳技术大学);Peking University(北京大学);Tongji University(同济大学);CASIA(中国科学院自动化所);Tencent(腾讯);UCLA(加州大学洛杉矶分校);Westlake University(西湖大学);NTU(国立大学)
HSSBench: Benchmarking Humanities and Social Sciences Ability for Multimodal Large Language Models
National Engineering Research Center for Software Engineering, Peking University(北京大学软件工程国家工程研究中心);Institute of Artificial Intelligence, China Telecom (TeleAI)(中国电信人工智能研究院);Tsinghua University(清华大学);Chinese Academy of Sciences(中国科学院);University of British Columbia(不列颠哥伦比亚大学);Renmin University of China(中国人民大学)
EscapeCraft: A 3D Room Escape Environment for Benchmarking Complex Multimodal Reasoning Ability
Dept. of Comp. Sci. & Tech., Institute for AI, Tsinghua University, Beijing, China(计算机科学与技术系,人工智能研究院,清华大学,北京,中国);Institute for AI Industry Research (AIR), Tsinghua University, Beijing, China(人工智能产业研究院(AIR),清华大学,北京,中国);School of Management, Fudan University, Shanghai, China(管理学院,复旦大学,上海,中国)
Enhancing the Reasoning Ability of Multimodal Large Language Models via Mixed Preference Optimization
OpenGVLab(开放智能机器研究实验室);Shanghai AI Laboratory(上海人工智能实验室);Fudan University(复旦大学);Nanjing University(南京大学);The Chinese University of Hong Kong(香港中文大学);Tsinghua University(清华大学);SenseTime Research(商汤科技研究院)
EgoBlind: Towards Egocentric Visual Assistance for the Blind
National University of Singapore(新加坡国立大学);Communication University of China(中国传媒大学);University of Science and Technology of China(中国科学技术大学);Hefei University of Technoloy(合肥工业大学)