arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2603.06043cs.CV
收藏

通过理解学习生成:基于理解的内在奖励用于统一多模态模型

Learning to Generate via Understanding: Understanding-Driven Intrinsic Rewarding for Unified Multimodal Models

Institute of Computing Technology, Chinese Academy of Sciences(中国科学院计算技术研究所);Peking University(北京大学);University of Chinese Academy of Sciences(中国科学院大学);Sun Yat-sen University(中山大学);Baidu Inc.(百度公司)

Jiadong Pan, Liang Li, Yuxin Peng, Yu-Ming Tang, Shuohuan Wang, Yu Sun, Hua Wu, Qingming Huang, Haifeng Wang

AI总结:

本文提出GvU机制,通过内在奖励提升统一多模态模型的生成能力,同时增强其视觉理解能力,缩小理解与生成之间的能力差距。

摘要:

最近,统一多模态模型(UMMs)在整合视觉理解和生成方面取得了显著进展,展示了在复杂文本到图像(T2I)任务中的强大潜力。尽管理论上有前景,但存在持续的能力差距:UMMs通常表现出优越的视觉理解能力,但相比之下生成能力较弱。这种差异主要源于理解与生成过程之间的内在解耦。虽然UMM可以准确解释细粒度的视觉细节,但往往难以从复杂的文本提示中生成语义连贯的图像。为了解决这一挑战,我们探索UMM内部的理解能力以提高生成质量。我们提出了一种基于token的内在文本-图像对齐奖励机制,称为GvU,使UMM同时充当教师和学生:它利用理解分支评估自己的输出以指导生成。在此基础上,我们设计了一种自监督强化学习框架,使UMM通过基于理解的内在奖励信号迭代改进其生成质量——无需外部监督。实验结果表明,我们的方法显著提高了UMM的生成能力,从而增强了其细粒度的视觉理解能力,缩小了UMM在视觉理解和生成之间的能力差距。

英文摘要:

Recently, unified multimodal models (UMMs) have made remarkable progress in integrating visual understanding and generation, demonstrating strong potential for complex text-to-image (T2I) tasks. Despite their theoretical promise, a persistent capability gap exists: UMMs typically exhibit superior visual understanding but comparatively weaker generative capabilities. This discrepancy arises largely from the intrinsic decoupling between the understanding and generation processes. While a UMM can accurately interpret fine-grained visual details, it often struggles to produce semantically coherent images from complex textual prompts. To address this challenge, we explore UMMs' internal understanding capability to enhance generation quality. We propose a token-level intrinsic text-image alignment reward mechanism, GvU, enabling the UMM to act simultaneously as teacher and student: it evaluates its own outputs using the understanding branch to guide the generations accordingly. Building upon this, we design a self-supervised reinforcement learning framework, allowing UMMs to iteratively improve their generation quality through understanding-based intrinsic reward signals--without reliance on external supervision. Experimental results show that our method substantially boosts UMMs' generation, which in turn strengthens their fine-grained visual understanding, narrowing the capability gap between UMMs' visual understanding and generation.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑