Ask, Solve, Generate: 通过自一致性奖励实现自我进化的统一多模态理解与生成
Ask, Solve, Generate: Self-Evolving Unified Multimodal Understanding and Generation via Self-Consistency Rewards
AI总结：
提出一个自我进化的训练框架，通过内部角色（提议者、求解者、生成者）和自一致性信号，无需人工标注或外部奖励模型，同时提升统一多模态模型的理解与生成能力。
摘要：
大多数支持视觉理解和图像生成的统一大型多模态模型（LMMs）仍然依赖于精心策划的后训练监督，例如人工标注、偏好标签或外部奖励模型。我们提出一个问题：一个统一的LMM能否仅使用未标注图像自主地提升这两种能力？我们提出了一个具有三个内部角色的自我进化训练框架：一个生成视觉问题的提议者，一个回答并评估问题的求解者，以及一个合成图像的生成者。训练仅使用自导出的一致性信号，无需人工标注、偏好标签或任务训练的外部奖励/评判模型。为了稳定学习，我们引入了求解者令牌熵（STE），一种基于令牌级预测不确定性的连续难度信号，即使在样本级一致性变得不可靠时仍然有用。对于图像生成，我们设计了一个多尺度内部评估方案，结合了问答保真度评分和循环一致性字幕。这创建了一个由求解者介导的耦合，其中更好的视觉理解使得更可靠的生成评估和更强的内部训练信号成为可能。该框架在基于扩散的BLIP3o、修正流BAGEL和自回归VARGPT-v1.1架构上保持了相同的角色分解、奖励逻辑和训练计划，仅需要每个骨干网络的原生提示和生成接口。在八个理解指标上，我们的方法持续优于相应的基础模型。在BAGEL上，它在MMMU上实现了+3.5%的绝对增益，并将GenEval图像生成性能从82%提高到85%。代码和模型已公开发布。
英文摘要：
Most unified large multimodal models (LMMs) that support both visual understanding and image generation still rely on curated post-training supervision, such as human annotations, preference labels, or external reward models. We ask whether a unified LMM can improve both abilities autonomously using only unlabeled images. We propose a self-evolving training framework with three internal roles: a Proposer that generates visual questions, a Solver that answers and evaluates them, and a Generator that synthesizes images. Training uses only self-derived consistency signals, without human annotations, preference labels, or task-trained external reward/judge models. To stabilize learning, we introduce Solver Token Entropy (STE), a continuous difficulty signal based on token-level prediction uncertainty that remains useful even when sample-level consistency becomes unreliable. For image generation, we design a multi-scale internal evaluation scheme that combines question-answer fidelity scoring with cycle-consistent captioning. This creates a solver-mediated coupling, where better visual understanding enables more reliable generation assessment and stronger internal training signals. The framework preserves the same role decomposition, reward logic, and training schedule across diffusion-based BLIP3o, rectified-flow BAGEL, and autoregressive VARGPT-v1.1 architectures, requiring only each backbone's native prompting and generation interface. Across eight understanding metrics, our method consistently improves over the corresponding base models. On BAGEL, it achieves a $+3.5\%$ absolute gain on MMMU and improves GenEval image generation performance from $82\%$ to $85\%$. Code and models are publicly released.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
