arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.09165cs.CV
收藏

文本到图像合成的综合条件化

All-in-One Conditioning for Text-to-Image Synthesis

University of Maryland, College Park（马里兰大学学院公园分校）

Hirunima Jayasekara, Chuong Huynh, Yixuan Ren, Christabel Acquaye, Abhinav Shrivastava

阅读 PDF
arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出基于场景图的零样本条件化机制，通过轻量级语言模型生成视觉条件，提升文本到图像合成的语义一致性和生成多样性。

摘要：

准确解读和视觉表示涉及多个对象、属性和空间关系的复杂提示是文本到图像合成中的关键挑战。尽管在生成逼真输出方面取得了最近的进步，但当前模型在处理复杂文本输入时往往难以保持语义一致性和结构连贯性。我们提出了一种新的方法，将文本到图像合成置于场景图结构框架内，旨在增强现有模型的组合能力。尽管先前的方法尝试通过使用预定义的布局图来解决这个问题，但这些刚性的约束往往限制了组合的灵活性和多样性。相反，我们引入了一种零样本、基于场景图的条件化机制，在推理过程中生成柔和的视觉指导。我们方法的核心是属性-大小-数量-位置（ASQL）条件器，它通过轻量级语言模型生成视觉条件，并通过推理时的优化引导扩散生成。这使模型能够在保持文本图像一致性的同时，支持轻量、连贯和多样化的图像合成。

英文摘要：

Accurate interpretation and visual representation of complex prompts involving multiple objects, attributes, and spatial relationships is a critical challenge in text-to-image synthesis. Despite recent advancements in generating photorealistic outputs, current models often struggle with maintaining semantic fidelity and structural coherence when processing intricate textual inputs. We propose a novel approach that grounds text-to-image synthesis within the framework of scene graph structures, aiming to enhance the compositional abilities of existing models. Eventhough, prior approaches have attempted to address this by using pre-defined layout maps derived from prompts, such rigid constraints often limit compositional flexibility and diversity. In contrast, we introduce a zero-shot, scene graph-based conditioning mechanism that generates soft visual guidance during inference. At the core of our method is the Attribute-Size-Quantity-Location (ASQL) Conditioner, which produces visual conditions via a lightweight language model and guides diffusion-based generation through inference-time optimization. This enables the model to maintain text-image alignment while supporting lightweight, coherent, and diverse image synthesis.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

↑