Despite the growing popularity of Multimodal Domain Generalization (MMDG) for enhancing model robustness, it remains unclear whether reported performance gains reflect genuine algorithmic progress or are artifacts of inconsistent evaluation protocols. Current research is fragmented, with studies varying significantly across datasets, modality configurations, and experimental settings. Furthermore, existing benchmarks focus predominantly on action recognition, often neglecting critical real-world challenges such as input corruptions, missing modalities, and model trustworthiness. This lack of standardization obscures a reliable assessment of the field's advancement. To address this issue, we introduce MMDG-Bench, the first unified and comprehensive benchmark for MMDG, which standardizes evaluation across six datasets spanning three diverse tasks: action recognition, mechanical fault diagnosis, and sentiment analysis. MMDG-Bench encompasses six modality combinations, nine representative methods, and multiple evaluation settings. Beyond standard accuracy, it systematically assesses corruption robustness, missing-modality generalization, misclassification detection, and out-of-distribution detection. With 7, 402 neural networks trained in total across 95 unique cross-domain tasks, MMDG-Bench yields five key findings: (1) under fair comparisons, recent specialized MMDG methods offer only marginal improvements over ERM baseline; (2) no single method consistently outperforms others across datasets or modality combinations; (3) a substantial gap to upper-bound performance persists, indicating that MMDG remains far from solved; (4) trimodal fusion does not consistently outperform the strongest bimodal configurations; and (5) all evaluated methods exhibit significant degradation under corruption and missing-modality scenarios, with some methods further compromising model trustworthiness.
Multimodal Clinical Benchmark for Emergency Care (MC-BEC): A Comprehensive Benchmark for Evaluating Foundation Models in Emergency Medicine
Harvard Medical School（哈佛医学院）；Stanford University（斯坦福大学）；Harvard John A. Paulson School of Engineering and Applied Sciences, Harvard University（哈佛大学约翰·A·保尔森工程与应用科学学院）
Similarity over Factuality: Are we making progress on multimodal out-of-context misinformation detection?
Information Technology Institute, Centre for Research & Technology, Hellas（希腊研究与技术中心信息技术研究所）；Department of Electrical & Computer Engineering, Aristotle University of Thessaloniki（亚里士多德大学电气与计算机工程系）
UniSAFE: A Comprehensive Benchmark for Safety Evaluation of Unified Multimodal Models
KAIST AI（KAIST人工智能研究院）；Department of Computer Science and Engineering, UNIST（UNIST计算机科学与工程系）；Department of Mathematical Sciences, KAIST（KAIST数学科学系）；University of Toronto（多伦多大学）；KAIST CS（KAIST计算机科学系）
MathVis-Fine: Aligning Visual Supervision with Necessity via Progressive Dependency-Guided Training for Multimodal Mathematical Reasoning
School of ECE, Peking University（北京大学电子与计算机工程学院）；College of Computer Science and Artificial Intelligence, Fudan University（复旦大学计算机科学与技术学院）；School of Software and Microelectronics, Peking University（北京大学软件与微电子学院）；Tencent Youtu Lab（腾讯优图实验室）
ComprehendEdit: A Comprehensive Dataset and Evaluation Framework for Multimodal Knowledge Editing
Harbin Institute of Technology（哈尔滨工业大学）；Pengcheng Laboratory（鹏城实验室）；Shenzhen University of Advanced Technology（深圳先进技术大学）；Northwestern Polytechnical University（西北工业大学）；Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）
MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Models
State Key Laboratory for Novel Software Technology, Nanjing University（南京大学新型软件技术国家重点实验室）；School of Intelligence Science and Technology, Nanjing University（南京大学智能科学与技术学院）；Tencent Youtu Lab（腾讯优图实验室）；Xiamen University（厦门大学）；CASIA（中国科学院自动化研究所）
DeepSport: A Multimodal Large Language Model for Comprehensive Sports Video Reasoning via Agentic Reinforcement Learning
Georgia Institute of Technology（佐治亚理工学院）；Rice University（Rice大学）；Johns Hopkins University（约翰霍普金斯大学）；University of California, Irvine（加州大学 Irvine分校）；University of California, Santa Barbara（加州大学圣巴巴拉分校）
INS-MMBench: A Comprehensive Benchmark for Evaluating LVLMs' Performance in Insurance
School of Computer Science Fudan University（复旦大学计算机科学学院）；Department of Computer Science University of Rochester（罗切斯特大学计算机科学系）；School of Economics Fudan University（复旦大学经济学院）
MuDD: A Multimodal Deception Detection Dataset and GSR-Guided Progressive Distillation for Non-Contact Deception Detection
School of Computer Science and Engineering, University of Electronic Science and Technology of China（电子科技大学计算机科学与工程学院）；School of Information and Software Engineering, University of Electronic Science and Technology of China（电子科技大学信息与软件工程学院）
MMRole: A Comprehensive Framework for Developing and Evaluating Multimodal Role-Playing Agents
Gaoling School of Artificial Intelligence, Renmin University of China（中国人民大学高瓴人工智能学院）；College of Information and Electrical Engineering, China Agricultural University（中国农业大学信息与电气工程学院）
CHIMERA Challenge Task 2 and 3: Response Subtypes Classification and Progression Survival Prediction in Bladder Cancer Patients using Multimodal Datasets
Radboud University Medical Center（拉德堡德大学医学中心）；Erasmus University Medical Center（伊拉斯姆斯大学医学中心）；Oncode Institute（Oncode研究所）；Canisius Wilhelmina Hospital（卡尼修斯·威廉米娜医院）；Tampere University（坦佩雷大学）；Tampere University Hospital（坦佩雷大学医院）
LMOD+: A Comprehensive Multimodal Dataset and Benchmark for Developing and Evaluating Multimodal Large Language Models in Ophthalmology
School of Medicine, Yale University（耶鲁大学医学院）；School of Computing, Australian National University（澳大利亚国立大学计算机学院）；School of Engineering, Imperial College London（伦敦帝国理工学院工程学院）；School of Computing, University of Georgia（佐治亚大学计算机学院）；Yong Loo Lin School of Medicine, National University of Singapore（新加坡国立大学杨秀隆医学学院）；National Eye Institute, National Institutes of Health（美国国立卫生研究院眼科研究所）；National Library of Medicine, National Institutes of Health（美国国立卫生研究院国家医学图书馆）；School of Computing Technologies, RMIT University（皇家墨尔本理工大学计算机技术学院）
Progressive Multimodal Interaction Network for Reliable Quantification of Fish Feeding Intensity in Aquaculture
National Innovation Center for Digital Fishery（国家数字渔业创新中心）；Key Laboratory of Smart Farming Technologies for Aquatic Animal and Livestock, Ministry of Agriculture and Rural Affairs（农业农村部水产动物与畜禽智慧养殖技术重点实验室）；State Key Laboratory of Efficient Utilization of Agricultural Water Resources（农业水资源高效利用全国重点实验室）；Beijing Engineering and Technology Research Center for Internet of Things in Agriculture（北京市农业物联网工程技术研究中心）；Key Laboratory of Digital Fisheries of Shandong Province（山东省数字渔业重点实验室）；College of Information and Electrical Engineering, China Agricultural University（中国农业大学信息与电气工程学院）
StanceFlip: A Comprehensive Multi-Dimensional Benchmark for Multimodal Conversational Stance Flipping Forecasting
College of Computer Science and Software Engineering, Shenzhen University（深圳大学计算机科学与软件学院）；Harbin Institute of Technology Shenzhen（哈尔滨工业大学（深圳））；City University of Macau（澳门城市大学）
SignIT: A Comprehensive Dataset and Multimodal Analysis for Italian Sign Language Recognition
Computer Science - University of Catania, Italy（计算机科学 - 卡塔尼亚大学，意大利）；Next Vision s.r.l., Spin-off of the University of Catania, Italy（2 Next Vision s.r.l.，卡塔尼亚大学分校，意大利）
LOKI: A Comprehensive Synthetic Data Detection Benchmark using Large Multimodal Models
Sun Yat-sen University（中山大学）；Shanghai AI Laboratory（上海人工智能实验室）；SenseTime Research（商汤科技研究院）；The Chinese University of Hong Kong（香港中文大学）；The Hong Kong University of Science and Technology（香港科技大学）；The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））
Flexible Multimodal Neuroimaging Fusion for Alzheimer's Disease Progression Prediction
Department of Computer Science and Engineering（计算机科学与工程系）；Department of Biomedical Informatics（生物医学信息学系）；Department of Neurology（神经病学系）；Translational Data Analytics Institute（转化数据分析研究所）
Next Token Prediction Towards Multimodal Intelligence: A Comprehensive Survey
Peking University（北京大学）；Beihang Univerisy（北京航空航天大学）；University of Hongkong（香港大学）；Shenzhen Institute of Advanced Technology, China Academy of Sciences（中国科学院深圳先进技术研究院）；Tsinghua University（清华大学）；M-A-P；The Chinese University of Hong Kong（香港中文大学）；Alibaba Group（阿里巴巴集团）；University of Cambridge（剑桥大学）；Microsoft Research（微软研究院）；UIUC（伊利诺伊大学厄巴纳-香槟分校）；Humanify Inc.（Humanify公司）；Zhejiang University（浙江大学）
Deep Learning in Multimodal Remote Sensing Data Fusion: A Comprehensive Review
Aerospace Information Research Institute, Chinese Academy of Sciences（中国科学院空天信息创新研究院）；University of Chinese Academy of Sciences（中国科学院大学）；Liaocheng University（聊城大学）
Empowering Multimodal LLMs with External Tools: A Comprehensive Survey
School of Automation Science and Engineering, Xi’an Jiaotong University（自动化科学与工程学院，西安交通大学）；Interdisciplinary Graduate Programme, Nanyang Technological University（跨学科研究生项目，南洋理工大学）；Lenovo Research, Lenovo（联想研究院，联想）；School of Computer Science and Technology, Xi’an Jiaotong University（计算机科学与技术学院，西安交通大学）；College of Computing and Data Science, Nanyang Technological University（计算与数据科学学院，南洋理工大学）
MMT-Bench: A Comprehensive Multimodal Benchmark for Evaluating Large Vision-Language Models Towards Multitask AGI
Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；Shanghai Jiao Tong University（上海交通大学）；The University of Hong Kong（香港大学）；The University of Adelaide（阿德莱德大学）；Zhejiang University（浙江大学）；Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）
