通过语义引导监督增强统一多模态模型的对齐
Enhancing Alignment for Unified Multimodal Models via Semantically-Grounded Supervision
AI总结：
本文提出Semantically-Grounded Supervision方法，通过构建视觉接地图和语义引导的破坏输入，解决统一多模态模型中的粒度不匹配和监督冗余问题，提升生成质量和跨模态对齐。
摘要：
统一多模态模型（UMMs）作为一种整合多模态理解和生成的统一建模框架，已显现巨大潜力。然而，当前生成训练范式存在固有局限。我们提出语义引导监督（SeGroS），一种细调框架，旨在解决UMMs中的粒度不匹配和监督冗余问题。其核心是提出新的视觉接地图，构建两种互补的监督信号。首先，我们制定语义视觉提示以补偿文本提示的稀疏性。其次，我们生成语义引导的破坏输入，通过限制重构损失到核心文本对齐区域，显式增强基于掩码的UMMs的监督。在GenEval、DPGBench和CompBench上的广泛评估表明，SeGroS显著提升了生成保真度和跨模态对齐能力，适用于各种UMM架构。
英文摘要：
Unified Multimodal Models (UMMs) have emerged as a promising paradigm that integrates multimodal understanding and generation within a unified modeling framework. However, current generative training paradigms suffer from inherent limitations. We present Semantically-Grounded Supervision (SeGroS), a fine-tuning framework designed to resolve the granularity mismatch and supervisory redundancy in UMMs. At its core, we propose a novel visual grounding map to construct two complementary supervision signals. First, we formulate semantic Visual Hints to compensate for the sparsity of text prompts. Second, we generate a semantically-grounded Corrupted Input to explicitly enhance the supervision of masking-based UMMs by restricting the reconstruction loss to core text-aligned regions. Extensive evaluations on GenEval, DPGBench, and CompBench demonstrate that SeGroS significantly improves generation fidelity and cross-modal alignment across various UMM architectures.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
