通过再生进行细化：扩大修改空间提升统一多模态模型中的图像细化
Refinement via Regeneration: Enlarging Modification Space Boosts Image Refinement in Unified Multimodal Models
AI总结：
本文提出通过再生进行细化的新框架，通过扩大修改空间提升统一多模态模型的图像细化效果，实验表明在多个基准测试中性能显著提升。
摘要：
统一多模态模型（UMMs）在单一框架内整合视觉理解和生成。对于文本到图像（T2I）任务，这种统一能力使UMMs能够在初始生成后细化输出，可能扩展性能上限。当前基于UMMs的细化方法主要遵循细化-编辑（RvE）范式，其中UMMs生成编辑指令以修改不匹配区域，同时保留匹配内容。然而，编辑指令通常仅粗略描述提示图像的不匹配，导致细化不完整。此外，像素级保留虽然必要，但限制了细化的有效修改空间。为解决这些限制，我们提出通过再生进行细化（RvR），一种新的框架，将细化重新公式化为条件图像再生，而不是编辑。与依赖编辑指令并强制严格内容保留不同，RvR根据目标提示和初始图像的语义标记再生图像，从而在更大的修改空间内实现更完整的语义对齐。大量实验表明RvR的有效性，改进了Geneval从0.78到0.91，DPGBench从84.02到87.21，以及UniGenBench++从61.53到77.41。
英文摘要：
Unified multimodal models (UMMs) integrate visual understanding and generation within a single framework. For text-to-image (T2I) tasks, this unified capability allows UMMs to refine outputs after their initial generation, potentially extending the performance upper bound. Current UMM-based refinement methods primarily follow a refinement-via-editing (RvE) paradigm, where UMMs produce editing instructions to modify misaligned regions while preserving aligned content. However, editing instructions often describe prompt-image misalignment only coarsely, leading to incomplete refinement. Moreover, pixel-level preservation, though necessary for editing, unnecessarily restricts the effective modification space for refinement. To address these limitations, we propose Refinement via Regeneration (RvR), a novel framework that reformulates refinement as conditional image regeneration rather than editing. Instead of relying on editing instructions and enforcing strict content preservation, RvR regenerates images conditioned on the target prompt and the semantic tokens of the initial image, enabling more complete semantic alignment with a larger modification space. Extensive experiments demonstrate the effectiveness of RvR, improving Geneval from 0.78 to 0.91, DPGBench from 84.02 to 87.21, and UniGenBench++ from 61.53 to 77.41.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
