arXivDaily arXiv每日学术速递 周一至周五更新
​通知:联邦节假日调休:哥伦布日 / 原住民日(Columbus Day / Indigenous Peoples' Day),今日(13日)无更新。
arXiv 2406.04032cs.CV

Zero-Painter:面向文本到图像合成的免训练布局控制

Zero-Painter: Training-Free Layout Control for Text-to-Image Synthesis

Picsart AI Research (PAIR)(Picsart人工智能研究院(PAIR));UT Austin(德克萨斯大学奥斯汀分校);Georgia Tech(佐治亚理工学院)

Marianna Ohanyan, Hayk Manukyan, Zhangyang Wang, Shant Navasardyan, Humphrey Shi

AI总结:

Zero-Painter提出免训练的布局条件文本到图像合成框架,通过PACA和ReGCA两阶段交叉注意力机制,实现对象与文本提示及掩码形状的精确对齐,在文本细节保留和掩码形状遵循上超越现有方法。

摘要:

我们提出Zero-Painter,一个新颖的免训练框架,用于布局条件文本到图像合成,有助于从文本提示创建详细且可控的图像。我们的方法利用对象掩码和单独描述,结合全局文本提示,生成高保真图像。Zero-Painter采用两阶段过程,包含我们新颖的Prompt-Adjusted Cross-Attention(PACA)和Region-Grouped Cross-Attention(ReGCA)模块,确保生成对象与文本提示及掩码形状精确对齐。大量实验表明,Zero-Painter在保留文本细节和遵循掩码形状方面超越了当前最先进的方法。

英文摘要:

We present Zero-Painter, a novel training-free framework for layout-conditional text-to-image synthesis that facilitates the creation of detailed and controlled imagery from textual prompts. Our method utilizes object masks and individual descriptions, coupled with a global text prompt, to generate images with high fidelity. Zero-Painter employs a two-stage process involving our novel Prompt-Adjusted Cross-Attention (PACA) and Region-Grouped Cross-Attention (ReGCA) blocks, ensuring precise alignment of generated objects with textual prompts and mask shapes. Our extensive experiments demonstrate that Zero-Painter surpasses current state-of-the-art methods in preserving textual details and adhering to mask shapes.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑