Recent advances in unified multimodal models (UMM) have demonstrated remarkable progress in both understanding and generation tasks. However, whether these two capabilities are genuinely aligned and integrated within a single model remains unclear. To investigate this question, we introduce GapEval, a bidirectional benchmark designed to quantify the gap between understanding and generation capabilities, and quantitatively measure the cognitive coherence of the two "unified" directions. Each question can be answered in both modalities (image and text), enabling a symmetric evaluation of a model's bidirectional inference capability and cross-modal consistency. Experiments reveal a persistent gap between the two directions across a wide range of UMMs with different architectures, suggesting that current models achieve only surface-level unification rather than deep cognitive convergence of the two. To further explore the underlying mechanism, we conduct an empirical study from the perspective of knowledge manipulation to illustrate the underlying limitations. Our findings indicate that knowledge within UMMs often remains disjoint. The capability emergence and knowledge across modalities are unsynchronized, paving the way for further exploration.
Quantifying & Modeling Multimodal Interactions: An Information Decomposition Framework
CMU(卡内基梅隆大学);Princeton University(普林斯顿大学);Columbia University(哥伦比亚大学);Harvard Medical School(哈佛医学院);Brigham and Women’s Hospital(布莱根妇女医院);University of Pennsylvania(宾夕法尼亚大学);University of Oregon(俄勒冈大学)
Quantifying and Mitigating Unimodal Biases in Multimodal Large Language Models: A Causal Perspective
Shanghai AI Laboratory(上海人工智能实验室);State Key Laboratory of General Artificial Intelligence, Peking University(北京大学通用人工智能国家重点实验室);School of Intelligence Science and Technology, Peking University(北京大学智能科学与技术学院);School of Computer Science, Fudan University(复旦大学计算机科学技术学院);Shanghai Artificial Intelligence Laboratory(上海人工智能实验室)
MEG-RAG: Quantifying Multi-modal Evidence Grounding for Evidence Selection in RAG
Institute for Clarity in Documentation(清晰文档研究所);Inria Paris-Rocquencourt(巴黎- Rocquencourt 国家信息与自动化研究所);Rajiv Gandhi University(拉吉夫·甘地大学);Tsinghua University(清华大学);Palmer Research Laboratories(帕勒尔研究实验室);Zhejiang University(浙江大学);Peking University(北京大学);University of New South Wales(新南威尔士大学);Macquarie University(麦考瑞大学);CSIRO’s Data61(CSIRO 数据61)
Mind the Gap No More: Achieving Zero-Gap Multimodal Integration via One Tokenizer
Research Center for Frontier Fundamental Studies, Zhejiang Lab(前沿基础研究研究中心,浙江实验室);Research Center for Scientific Data Hub, Zhejiang Lab(科学数据枢纽研究中心,浙江实验室)
MEDSYN: Benchmarking Multi-EviDence SYNthesis in Complex Clinical Cases for Multimodal Large Language Models
ETH Zurich(苏黎世联邦理工学院);MBZUAI(马克斯·普朗克人工智能研究所);Amazon(亚马逊);University of Oxford(牛津大学);University of Bern(伯尔尼大学);Imperial College London(伦敦帝国理工学院);Peking University(北京大学)
RNG: Reducing Multi-level Noise and Multi-grained Semantic Gap for Joint Multimodal Aspect-Sentiment Analysis
Institute of Information Engineering, Chinese Academy of Sciences(中国科学院信息工程研究所);School of Cyber Security, UCAS(中国科学院大学网络空间安全学院);Academy of Cyber(网络空间研究院)
RBF++: Quantifying and Optimizing Reasoning Boundaries across Measurable and Unmeasurable Capabilities for Chain-of-Thought Reasoning
Research Center for Social Computing and Interactive Robotics, Harbin Institute of Technology(社会计算与交互机器人研究中心,哈尔滨工业大学);Faculty of Computing, Harbin Institute of Technology(计算学院,哈尔滨工业大学);School of Computer Science and Engineering, Central South University(计算机科学与工程学院,中南大学);Chinese University of Hong Kong(香港中文大学);MMLab, The Chinese University of Hong Kong(香港中文大学MMLab)
How to Bridge the Gap between Modalities: Survey on Multimodal Large Language Model
College of Computer, National University of Defense Technology(国防科技大学计算机学院);School of Computer and Information Engineering, Hefei University of Technology(合肥工业大学计算机与信息工程学院)
Bridging the Gap between Synthetic and Authentic Images for Multimodal Machine Translation
Institute of Computing Technology, Chinese Academy of Sciences(中国科学院计算技术研究所);University of Chinese Academy of Sciences(中国科学院大学);Beijing Language and Culture University(北京语言大学)
COMET: Concept Space Dissection of the Modality Gap in Audio-Text Multimodal Contrastive Embeddings
School of Artificial Intelligence, Beijing University of Posts and Telecommunications(北京邮电大学人工智能学院);Centre for Vision, Speech, and Signal Processing (CVSSP), University of Surrey(Surrey 大学视觉、语音和信号处理中心)
Closing the Gap: Data-Centric Fine-Tuning of Vision Language Models for the Standardized Exam Questions
organization= Department of Computer Engineering, Middle East Technical University (METU) , city= Ankara , country= Türkiye;organization= METU-DTX Digital Transformation \& Innovation Centre, METU , city= Ankara , country= Türkiye