arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2602.02140cs.CL
收藏

量化统一多模态模型中理解与生成之间的差距

Quantifying the Gap between Understanding and Generation within Unified Multimodal Models

University of Maryland(马里兰大学);University of Waterloo(滑铁卢大学);MBZUAI

Chenlong Wang, Yuhang Chen, Zhihan Hu, Dongping Chen, Wenhu Chen, Sarah Wiegreffe, Tianyi Zhou

AI总结:

本研究通过GapEval基准量化统一多模态模型中理解与生成能力的差距,揭示当前模型仅实现表面层次的统一,而非深度认知融合。

摘要:

近年来,统一多模态模型(UMM)在理解和生成任务上取得了显著进展。然而,这两种能力是否在单一模型中真正对齐和整合仍不明确。为探讨这个问题,我们引入了GapEval,一个双向基准,旨在量化理解和生成能力之间的差距,并定量测量两种

英文摘要:

Recent advances in unified multimodal models (UMM) have demonstrated remarkable progress in both understanding and generation tasks. However, whether these two capabilities are genuinely aligned and integrated within a single model remains unclear. To investigate this question, we introduce GapEval, a bidirectional benchmark designed to quantify the gap between understanding and generation capabilities, and quantitatively measure the cognitive coherence of the two "unified" directions. Each question can be answered in both modalities (image and text), enabling a symmetric evaluation of a model's bidirectional inference capability and cross-modal consistency. Experiments reveal a persistent gap between the two directions across a wide range of UMMs with different architectures, suggesting that current models achieve only surface-level unification rather than deep cognitive convergence of the two. To further explore the underlying mechanism, we conduct an empirical study from the perspective of knowledge manipulation to illustrate the underlying limitations. Our findings indicate that knowledge within UMMs often remains disjoint. The capability emergence and knowledge across modalities are unsynchronized, paving the way for further exploration.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑