GoT：释放多模态大语言模型在视觉生成与编辑中的推理能力
GoT: Unleashing Reasoning Capability of Multimodal Large Language Model for Visual Generation and Editing
AI总结：
本文提出 GoT 范式，通过显式语言推理链引导图像生成与编辑，并构建超 900 万样本数据集及融合 Qwen2.5-VL 与扩散模型的统一框架，显著提升任务表现与交互可控性。
摘要：
当前的图像生成与编辑方法主要将文本提示作为直接输入处理，而不会对视觉构图和显式操作进行推理。我们提出了 Generation Chain-of-Thought（GoT，生成式思维链），这是一种新颖范式，使生成与编辑能够在输出图像之前经过显式的语言推理过程。该方法将传统的文本到图像生成与编辑转化为一个由推理引导的框架，用于分析语义关系和空间布局。我们定义了 GoT 的形式化表述，并构建了包含超过 900 万个样本的大规模 GoT 数据集，这些样本带有捕捉语义—空间关系的详细推理链。为利用 GoT 的优势，我们实现了一个统一框架，将用于推理链生成的 Qwen2.5-VL 与由我们新颖的语义—空间引导模块增强的端到端扩散模型相结合。实验表明，我们的 GoT 框架在生成和编辑任务上均取得优异性能，相较基线有显著提升。此外，我们的方法支持交互式视觉生成，允许用户显式修改推理步骤以进行精确的图像调整。GoT 为推理驱动的视觉生成与编辑开创了新方向，可生成更符合人类意图的图像。为促进未来研究，我们在 https://github.com/rongyaofang/GoT 公开了数据集、代码和预训练模型。
英文摘要：
Current image generation and editing methods primarily process textual prompts as direct inputs without reasoning about visual composition and explicit operations. We present Generation Chain-of-Thought (GoT), a novel paradigm that enables generation and editing through an explicit language reasoning process before outputting images. This approach transforms conventional text-to-image generation and editing into a reasoning-guided framework that analyzes semantic relationships and spatial arrangements. We define the formulation of GoT and construct large-scale GoT datasets containing over 9M samples with detailed reasoning chains capturing semantic-spatial relationships. To leverage the advantages of GoT, we implement a unified framework that integrates Qwen2.5-VL for reasoning chain generation with an end-to-end diffusion model enhanced by our novel Semantic-Spatial Guidance Module. Experiments show our GoT framework achieves excellent performance on both generation and editing tasks, with significant improvements over baselines. Additionally, our approach enables interactive visual generation, allowing users to explicitly modify reasoning steps for precise image adjustments. GoT pioneers a new direction for reasoning-driven visual generation and editing, producing images that better align with human intent. To facilitate future research, we make our datasets, code, and pretrained models publicly available at https://github.com/rongyaofang/GoT.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
