SRUM:细粒度自奖励用于统一多模态模型
SRUM: Fine-Grained Self-Rewarding for Unified Multimodal Models
AI总结:
SRUM通过内部评估模块对生成模块进行自奖励,提升统一多模态模型的视觉生成能力,实验证明在T2I-CompBench和T2I-ReasonBench上性能显著提升。
摘要:
近年来,统一多模态模型(UMMs)在视觉-语言生成与理解方面取得了显著进展。然而,模型强大的视觉理解能力往往无法迁移到视觉生成:它可能正确判断提示图像对齐,但无法从相同提示生成忠实图像。本文提出SRUM,一种自奖励后训练框架,直接适用于各种设计的UMMs。SRUM创建了一个反馈循环,使模型自身的理解模块充当内部“评估器”,通过提供纠正信号来改进生成,无需额外人工标注数据或外部奖励模型。为了提供全面反馈,SRUM采用全局-局部双奖励系统:全局奖励确保整体视觉语义和布局,局部奖励细化细粒度、对象级保真度。SRUM在T2I-CompBench上将性能从82.18提升至88.37,在T2I-ReasonBench上从43.82提升至46.75。整体而言,本文的工作建立了一种强大的范式,使UMM的理解模块能够通过自奖励引导并增强自身的生成能力。
英文摘要:
Recently, remarkable progress has been made in Unified Multimodal Models (UMMs), which integrate vision-language generation and understanding capabilities within a single framework. However, a model's strong visual understanding often fails to transfer to visual generation: it may correctly judge prompt-image alignment while failing to generate a faithful image from the same prompt. This raises a compelling question: Can a model improve itself by using its understanding module to reward its generation module? We introduce SRUM, a self-rewarding post-training framework directly applicable to existing UMMs of various designs. SRUM creates a feedback loop where the model's own understanding module acts as an internal ``evaluator'', providing corrective signals to improve generation without additional human-labeled data or external reward models. To provide comprehensive feedback, SRUM uses a global-local dual reward system: a \textbf{global reward} ensures overall visual semantics and layout, while a \textbf{local reward} refines fine-grained, object-level fidelity. SRUM shows strong generalization, boosting performance on T2I-CompBench from 82.18 to \textbf{88.37} and on T2I-ReasonBench from 43.82 to \textbf{46.75}. Overall, our work establishes a powerful paradigm for enabling a UMM's understanding module to guide and enhance its own generation via self-rewarding.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。