arXiv 2509.069452025-09-09cs.CVcs.AIcs.CLcs.LG

交错推理以改进文本到图像生成

Interleaving Reasoning for Better Text-to-Image Generation

East China Normal University（华东师范大学）；The Chinese University of Hong Kong（香港中文大学）；Xiaohongshu Inc.（小红书公司）；University of California, Los Angeles（加州大学洛杉矶分校）；Zhejiang University（浙江大学）；University of Oxford（牛津大学）

Wenxuan Huang, Shuang Chen, Zheyong Xie, Shaosheng Cao, Shixiang Tang, Yufan Shen, Qingyu Yin, Wenbo Hu, Xiaoman Wang, Yuntian Tang, Junbo Qiao, Yue Guo, Yao Hu… 展开作者

Wenxuan Huang, Shuang Chen, Zheyong Xie, Shaosheng Cao, Shixiang Tang, Yufan Shen, Qingyu Yin, Wenbo Hu, Xiaoman Wang, Yuntian Tang, Junbo Qiao, Yue Guo, Yao Hu, Zhenfei Yin, Philip Torr, Yu Cheng, Wanli Ouyang, Shaohui Lin

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

针对统一多模态模型在指令遵循和细节保留上的不足，提出交错推理生成（IRG）框架，在文本思考与图像合成间交替以细化生成，并设计IRGL训练策略及IRGL-300K数据集，在多个基准上取得5-10点绝对提升。

摘要：

统一的多模态理解与生成模型近期在图像生成能力上取得了显著进步，但在指令遵循和细节保留方面，与诸如GPT-4o等将理解与生成紧密耦合的系统相比，仍存在较大差距。受交错推理近期进展的启发，我们探索这种推理能否进一步改进文本到图像（T2I）生成。我们引入了交错推理生成（IRG）框架，该框架在基于文本的思考与图像合成之间交替进行：模型首先产生基于文本的思考以指导初始图像，然后反思结果以细化细节、视觉质量和美学，同时保持语义。为了有效训练IRG，我们提出了交错推理生成学习（IRGL），其针对两个子目标：（1）加强初始思考-生成阶段以建立核心内容和基础质量，（2）在后续图像中实现高质量的文本反思及对这些改进的忠实执行。我们整理了IRGL-300K数据集，该数据集组织为六种分解学习模式，共同涵盖基于文本的思考学习以及完整的思考-图像轨迹。从一个原生输出交错文本-图像输出的统一基础模型出发，我们的两阶段训练首先构建稳健的思考和反思，然后在完整的思考-图像轨迹数据中高效调整IRG流水线。大量实验展示了最先进的性能，在GenEval、WISE、TIIF、GenAI-Bench和OneIG-EN上取得了5-10个百分点的绝对提升，同时在视觉质量和细粒度保真度上也有显著改进。代码、模型权重和数据集将在https://github.com/Osilly/Interleaving-Reasoning-Generation 发布。

英文摘要：

Unified multimodal understanding and generation models recently have achieve significant improvement in image generation capability, yet a large gap remains in instruction following and detail preservation compared to systems that tightly couple comprehension with generation such as GPT-4o. Motivated by recent advances in interleaving reasoning, we explore whether such reasoning can further improve Text-to-Image (T2I) generation. We introduce Interleaving Reasoning Generation (IRG), a framework that alternates between text-based thinking and image synthesis: the model first produces a text-based thinking to guide an initial image, then reflects on the result to refine fine-grained details, visual quality, and aesthetics while preserving semantics. To train IRG effectively, we propose Interleaving Reasoning Generation Learning (IRGL), which targets two sub-goals: (1) strengthening the initial think-and-generate stage to establish core content and base quality, and (2) enabling high-quality textual reflection and faithful implementation of those refinements in a subsequent image. We curate IRGL-300K, a dataset organized into six decomposed learning modes that jointly cover learning text-based thinking, and full thinking-image trajectories. Starting from a unified foundation model that natively emits interleaved text-image outputs, our two-stage training first builds robust thinking and reflection, then efficiently tunes the IRG pipeline in the full thinking-image trajectory data. Extensive experiments show SoTA performance, yielding absolute gains of 5-10 points on GenEval, WISE, TIIF, GenAI-Bench, and OneIG-EN, alongside substantial improvements in visual quality and fine-grained fidelity. The code, model weights and datasets will be released in: https://github.com/Osilly/Interleaving-Reasoning-Generation .

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。