分而治之：语言模型能够为组合式文本到图像生成进行规划与自校正
Divide and Conquer: Language Models can Plan and Self-Correct for Compositional Text-to-Image Generation
AI总结：
针对复杂文本提示下文本到图像生成可控性不足的问题，提出以LLM智能体为核心的无训练方法CompAgent，通过分治策略分解组合任务、规划工具生成并自校正，在T2I-CompBench上实现超10%的性能提升。
摘要：
尽管文本到图像模型在生成高质量图像方面已取得显著进展，但在面对复杂文本提示时，这些方法仍难以确保文本提示对图像的可控性，尤其是在保留物体属性和关系方面。本文提出CompAgent，一种无需训练的组合式文本到图像生成方法，其核心为大语言模型（LLM）智能体。CompAgent的基本思想基于分而治之的方法论。给定一个包含多个概念（包括物体、属性和关系）的复杂文本提示，LLM智能体首先对其进行分解，包括提取单个物体及其相关属性，并预测连贯的场景布局。随后这些单个物体可被独立处理。接着，智能体通过分析文本进行推理，规划并使用工具将这些孤立的物体组合起来。最后，我们在智能体中引入验证和人类反馈机制，以进一步纠正潜在的属性错误并优化生成的图像。在LLM智能体的引导下，我们提出了一个无需微调的多概念定制模型和一个布局到图像生成模型作为概念组合的工具，以及一种局部图像编辑方法作为与智能体交互以进行验证的工具。场景布局在这些工具中控制图像生成过程，以防止多个物体之间出现混淆。大量实验证明了我们的方法在组合式文本到图像生成方面的优越性：CompAgent在T2I-CompBench（一个面向开放世界组合式文本到图像生成的综合基准）上实现了超过10%的性能提升。对各类相关任务的扩展也表明了CompAgent在潜在应用中的灵活性。
英文摘要：
Despite significant advancements in text-to-image models for generating high-quality images, these methods still struggle to ensure the controllability of text prompts over images in the context of complex text prompts, especially when it comes to retaining object attributes and relationships. In this paper, we propose CompAgent, a training-free approach for compositional text-to-image generation, with a large language model (LLM) agent as its core. The fundamental idea underlying CompAgent is premised on a divide-and-conquer methodology. Given a complex text prompt containing multiple concepts including objects, attributes, and relationships, the LLM agent initially decomposes it, which entails the extraction of individual objects, their associated attributes, and the prediction of a coherent scene layout. These individual objects can then be independently conquered. Subsequently, the agent performs reasoning by analyzing the text, plans and employs the tools to compose these isolated objects. The verification and human feedback mechanism is finally incorporated into our agent to further correct the potential attribute errors and refine the generated images. Guided by the LLM agent, we propose a tuning-free multi-concept customization model and a layout-to-image generation model as the tools for concept composition, and a local image editing method as the tool to interact with the agent for verification. The scene layout controls the image generation process among these tools to prevent confusion among multiple objects. Extensive experiments demonstrate the superiority of our approach for compositional text-to-image generation: CompAgent achieves more than 10\% improvement on T2I-CompBench, a comprehensive benchmark for open-world compositional T2I generation. The extension to various related tasks also illustrates the flexibility of our CompAgent for potential applications.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
