Re-Align:基于结构化推理的对齐方法用于上下文图像生成与编辑
Re-Align: Structured Reasoning-guided Alignment for In-Context Image Generation and Editing
AI总结:
Re-Align通过结构化推理引导的对齐方法,提升上下文图像生成与编辑任务的性能。
摘要:
上下文图像生成与编辑(ICGE)允许用户通过交错的图像-文本提示指定视觉概念,要求对用户意图有精确的理解和忠实的执行。尽管最近的统一多模态模型表现出有前途的理解能力,但这些优势往往无法有效转移到图像生成中。我们引入了Re-Align,一种统一的框架,通过结构化推理引导的对齐来弥合理解和生成之间的差距。其核心是上下文链式推理(IC-CoT),一种结构化推理范式,它将语义指导与参考关联解耦,提供清晰的文本目标,并缓解参考图像之间的混淆。此外,Re-Align引入了一种有效的强化学习训练方案,利用替代奖励来衡量结构化推理文本与生成图像之间的对齐程度,从而提高模型在ICGE任务上的整体性能。广泛的实验验证了Re-Align在上下文图像生成和编辑任务上均优于同等规模和资源的竞争方法。
英文摘要:
In-context image generation and editing (ICGE) enables users to specify visual concepts through interleaved image-text prompts, demanding precise understanding and faithful execution of user intent. Although recent unified multimodal models exhibit promising understanding capabilities, these strengths often fail to transfer effectively to image generation. We introduce Re-Align, a unified framework that bridges the gap between understanding and generation through structured reasoning-guided alignment. At its core lies the In-Context Chain-of-Thought (IC-CoT), a structured reasoning paradigm that decouples semantic guidance and reference association, providing clear textual target and mitigating confusion among reference images. Furthermore, Re-Align introduces an effective RL training scheme that leverages a surrogate reward to measure the alignment between structured reasoning text and the generated image, thereby improving the model's overall performance on ICGE tasks. Extensive experiments verify that Re-Align outperforms competitive methods of comparable model scale and resources on both in-context image generation and editing tasks.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。