发表机构
S-Lab, Nanyang Technological University; SenseTime Research（新加坡南洋理工大学S-Lab; 商汤科技研究院）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本文研究无预训练视觉先验的原生统一多模态模型，从表示、任务、系统层面揭示视觉理解与生成的协同效应，发现适当设计可将二者共存转化为协同，且端到端模型优于规划器-执行器流水线。
AI 中文摘要
统一多模态模型（UMMs）可在单个模型中同时执行视觉理解与生成任务，但功能统一并不必然带来学习协同：两个目标可能相互促进、争夺容量，或仅单纯共存。本文在无预训练视觉先验的受控、结构原生设置下，从表示、任务、系统三个层面研究二者关系。在表示层面，我们发现每个目标都为对方提供有用信号：生成任务丰富理解所学习的视觉特征，而理解任务强化生成所需的视觉-语言对齐；但当两个目标被强制通过同一计算路径时，其中一个往往会占据主导。任务解耦架构通过专门处理冲突的视觉计算同时保留语义交互，可避免这种非对称退化。在任务层面，通过三个案例研究，我们发现当理解与生成任务依赖共享知识时，会出现正向双向迁移。在系统层面，我们表明端到端UMMs在明确需要图像理解与生成的复杂任务上，优于匹配的规划器-执行器流水线。综上，这些结果表明UMMs的价值超越了统一接口：适当的专门化、共享任务知识与端到端优化可将共存转化为协同。
英文摘要
While unified multimodal models (UMMs) jointly perform visual understanding and generation within a single model, functional unification does not guarantee learning synergy: the two objectives may reinforce each other, compete for capacity, or merely coexist. We investigate their relationship at the representation, task, and system levels in a controlled, structurally native setting without pretrained vision priors. At the representation level, we find that each objective provides useful signal to the other: generation enriches the visual features learned for understanding, while understanding strengthens vision--language alignment for generation. However, when both objectives are forced through the same computation path, one tends to dominate. A task-decoupled architecture that specializes conflicting visual computation while preserving semantic interaction avoids this asymmetric degradation. At the task level, through three case studies, we find positive bidirectional transfer when understanding and generation tasks rely on shared knowledge. At the system level, we show that an end-to-end UMM outperforms a matched planner--executor pipeline on complex tasks that explicitly require both image understanding and generation. Together, these results show that the value of UMMs extends beyond a unified interface: appropriate specialization, shared task knowledge, and end-to-end optimization can turn coexistence into synergy.
