LayoutLLM-T2I：从 LLM 中引出布局指导以进行文本到图像生成
LayoutLLM-T2I: Eliciting Layout Guidance from LLM for Text-to-Image Generation
AI总结：
针对文本到图像生成中的空间与数量错位问题，本文提出 LayoutLLM-T2I，利用 LLM 上下文学习自动规划布局，并通过细粒度对象交互扩散生成高保真对齐图像。
摘要：
在文本到图像生成领域，Stable Diffusion 近期取得的显著进展使生成丰富多样的新颖照片级真实图像成为可能。然而，当前模型在复杂自然场景中仍面临错位问题（例如，空间关系理解存在缺陷以及计数失败），这阻碍了高保真度文本到图像生成。尽管近期研究已尝试通过提供细粒度指导（如草图和涂鸦）来提升可控性，但由于用户必须手动提供此类指导信息，该问题尚未从根本上得到解决。在本工作中，我们致力于在无需任何指导的情况下，合成与给定文本提示在语义上对齐的高保真图像。为此，我们提出一种由粗到细的范式，以实现布局规划和图像生成。具体而言，我们首先基于 Large Language Models（大语言模型）通过上下文学习，在给定文本提示的条件下生成粗粒度布局。随后，我们提出一种细粒度对象交互扩散方法，在提示和自动生成布局的条件下合成高保真度图像。大量实验表明，我们提出的方法在布局和图像生成方面优于最先进模型。我们的代码和设置可在 https://layoutllm-t2i.github.io 获取。
英文摘要：
In the text-to-image generation field, recent remarkable progress in Stable Diffusion makes it possible to generate rich kinds of novel photorealistic images. However, current models still face misalignment issues (e.g., problematic spatial relation understanding and numeration failure) in complex natural scenes, which impedes the high-faithfulness text-to-image generation. Although recent efforts have been made to improve controllability by giving fine-grained guidance (e.g., sketch and scribbles), this issue has not been fundamentally tackled since users have to provide such guidance information manually. In this work, we strive to synthesize high-fidelity images that are semantically aligned with a given textual prompt without any guidance. Toward this end, we propose a coarse-to-fine paradigm to achieve layout planning and image generation. Concretely, we first generate the coarse-grained layout conditioned on a given textual prompt via in-context learning based on Large Language Models. Afterward, we propose a fine-grained object-interaction diffusion method to synthesize high-faithfulness images conditioned on the prompt and the automatically generated layout. Extensive experiments demonstrate that our proposed method outperforms the state-of-the-art models in terms of layout and image generation. Our code and settings are available at https://layoutllm-t2i.github.io.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
