MLLM-4D: Towards Visual-based Spatial-Temporal Intelligence
University of Macau（澳门大学）；Xi'an Jiaotong University（西安交通大学）；The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））；GVC Lab, Great Bay University（大湾大学GVC实验室）
Xingyilang Yin, Chengzhengxu Li, Jiahao Chang, Chi-Man Pun, Xiaodong Cun
Humans are born with vision-based 4D spatial-temporal intelligence, which enables us to perceive and reason about the evolution of 3D space over time from purely visual inputs. Despite its importance, this capability remains a significant bottleneck for current multimodal large language models (MLLMs). To tackle this challenge, we introduce MLLM-4D, a comprehensive framework designed to bridge the gaps in training data curation and model post-training for spatiotemporal understanding and reasoning. On the data front, we develop a cost-efficient data curation pipeline that repurposes existing stereo video datasets into high-quality 4D spatiotemporal instructional data. This results in the MLLM4D-2M and MLLM4D-R1-30k datasets for Supervised Fine-Tuning (SFT) and Reinforcement Fine-Tuning (RFT), alongside MLLM4D-Bench for comprehensive evaluation. Regarding model training, our post-training strategy establishes a foundational 4D understanding via SFT and further catalyzes 4D reasoning capabilities by employing Group Relative Policy Optimization (GRPO) with specialized Spatiotemporal Chain of Thought (ST-CoT) prompting and Spatiotemporal reward functions (ST-reward) without involving the modification of architecture. Extensive experiments demonstrate that MLLM-4D achieves state-of-the-art spatial-temporal understanding and reasoning capabilities from purely 2D RGB inputs. Project page: https://github.com/GVCLab/MLLM-4D.
Creation-MMBench: Assessing Context-Aware Creative Intelligence in MLLM
Zhejiang University（浙江大学）；Shanghai AI Laboratory（上海人工智能实验室）；Tongji University（同济大学）；Nanjing University（南京大学）；East China Normal University（华东师范大学）；Shanghai Jiaotong University（上海交通大学）；The Chinese University of Hong Kong（香港中文大学）
Tell Model Where to Look: Mitigating Hallucinations in MLLMs by Vision-Guided Attention
School of Computer Science and Technology, Beijing Institute of Technology（北京理工大学计算机科学与技术学院）；Zhongguancun Academy（中关村学院）；Southeast Academy of Information Technology, Beijing Institute of Technology（北京理工大学信息科技东南学院）；Zhongguancun Laboratory（中关村实验室）
Open Eyes, Then Reason: Fine-grained Visual Mathematical Understanding in MLLMs
Australian Institute for Machine Learning, University of Adelaide（阿德莱德大学澳大利亚机器学习研究所）；Georgia Institute of Technology（佐治亚理工学院）；Nanjing University of Science and Technology（南京理工大学）；University of Oxford（牛津大学）；The Ohio State University（俄亥俄州立大学）
Mitigating Visual Degradation in MLLMs via Spatial-Spectral Visual Anchor Learning
China University of Petroleum (East China)（中国石油大学（华东））；Shanghai Jiao Tong University（上海交通大学）；Wuhan University（武汉大学）；Great Wall Motor（长城汽车）；Nanyang Technological University（南洋理工大学）；The University of Hong Kong（香港大学）
ERA: Entropy-Guided Visual Token Pruning with Rectified Attention for Efficient MLLMs
School of Future Technology, Dalian University of Technology（大连理工大学未来技术学院）；S-Lab, Nanyang Technological University（南洋理工大学S-Lab）；OPPO Research Institute（OPPO研究院）；Department of Computing, The Hong Kong Polytechnic University（香港理工大学计算机系）
Unveiling the Ignorance of MLLMs: Seeing Clearly, Answering Incorrectly
Hong Kong University of Science and Technology（香港科技大学）；School of AI, Shanghai Jiao Tong University（上海交通大学人工智能学院）；Beijing University of Posts and Telecommunications（北京邮电大学）；Peking University（北京大学）；Everlyn AI；Youtu Lab, Tencent（腾讯优图实验室）；Beijing Academy of Artificial Intelligence（北京人工智能研究院）；University of Central Florida（中佛罗里达大学）
DisCo: Towards Distinct and Coherent Visual Encapsulation in Video MLLMs
University of Chinese Academy of Sciences（中国科学院大学）；The University of Hong Kong（香港大学）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；Shanghai Innovation Institute（上海创新研究院）；Fudan University（复旦大学）
VisualQuest: A Benchmark for Abstract Visual Reasoning in MLLMs
School of Computer Science and Technology, Dalian University of Technology, Dalian, China（大连理工大学计算机科学与技术学院）；School of Foreign Languages, Dalian University of Technology, Dalian, China（大连理工大学外语学院）；School of Computer Science and Technology, Xinjiang Normal University, Urumqi, China（新疆师范大学计算机科学与技术学院）
School of Software, Tsinghua University（清华大学软件学院）；BNRist, Tsinghua University（清华大学BNRist）；Department of Automation, Tsinghua University（清华大学自动化系）
Multimodal Large Language Models (MLLMs) have recently demonstrated strong performance across a wide range of vision-language tasks, garnering significant attention in the computer vision. However, their efficient deployment remains a substantial challenge due to high computational costs and memory requirements. Recognizing the redundancy of information within the vision modality, recent studies have explored methods for compressing visual tokens in MLLMs to enhance efficiency in a training-free manner. Despite their effectiveness, existing methods like Fast rely on the attention between visua…
Customizing Visual Emotion Evaluation for MLLMs: An Open-vocabulary, Multifaceted, and Scalable Approach
Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）；VCIP & TMCC & DISSec, College of Computer Science, Nankai University（南开大学计算机学院）；Department of Psychological and Cognitive Sciences, Tsinghua University（清华大学心理学与认知科学系）；University of Chinese Academy of Sciences（中国科学院大学）
Scenes as an Attack Surface: Uncovering Scene-Grounded Safety Failures in MLLMs
Nanjing University（南京大学）；The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；China Academy of Launch Vehicle Technology（中国运载火箭技术研究院）；Beihang University（北京航空航天大学）
RedundancyLens: Revealing and Exploiting Visual Token Processing Redundancy for Efficient Decoder-Only MLLMs
South China University of Technology（华南理工大学）；Intsig Information Co., Ltd.（Intsig信息有限公司）；INTSIG-SCUT Joint Lab on Document Analysis and Recognition（INTSIG-SCUT文档分析与识别联合实验室）
ViSTR-Bench: Can MLLMs Reason from Continuous Visual Cues in Dynamic Scenes?
School of Artificial Intelligence, Beihang University（北京航空航天大学人工智能学院）；Zhongguancun Academy（中关村科学城）；School of Engineering, Westlake University（西湖大学工学院）；School of Transportation Science and Engineering, Beihang University（北京航空航天大学交通科学与工程学院）
The Coherence Trap: When MLLM-Crafted Narratives Exploit Manipulated Visual Contexts
School of Software Engineering, Xi’an Jiaotong University（西安交通大学软件工程学院）；School of Computer Science and Information Engineering, Hefei University of Technology（合肥工业大学计算机科学与信息工程学院）；CSIRO（澳大利亚联邦科学与工业研究组织）；Northwestern Polytechnical University（西北工业大学）；University of Macau（澳门大学）
Vision-Centric Activation and Coordination for Multimodal Large Language Models
MoE Key Lab of Artificial Intelligence, Shanghai Jiao Tong University（人工智能MoE实验室，上海交通大学）；Ant Group（蚂蚁集团）；Ningbo Institute of Digital Twin, Eastern Institute of Technology, Ningbo（宁波数字孪生研究所，东部技术研究所，宁波）
