arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2604.18258cs.CVcs.AI
收藏

通过组合提示分解实现长文本到图像生成

Long-Text-to-Image Generation via Compositional Prompt Decomposition

Peking University(北京大学);Harvard University(哈佛大学)

Jen-Yuan Huang, Tong Lin, Yilun Du

AI总结:

本文提出PRISM方法,通过轻量模块提取长提示的组成部分,使预训练T2I模型能处理长序列输入,实现对复杂场景的建模,优于基线模型。

摘要:

尽管现代文本到图像(T2I)模型在生成复杂提示的图像方面表现出色,但当输入为描述性段落时,它们难以捕捉关键细节。这种限制源于训练分布中简短标题的普遍性。现有方法试图通过在长提示上微调T2I模型或投影超大输入到正常提示空间来弥合这一差距。我们提出Prompt Refraction for Intricate Scene Modeling(PRISM),一种组合方法,使预训练T2I模型能够处理长序列输入。PRISM使用轻量模块从长提示中提取组成部分表示。T2I模型对每个组件独立进行噪声预测,然后通过能量基联合将输出合并到单个去噪步骤中。我们评估了PRISM在多种模型架构上的性能,显示其与在相同训练数据上微调的模型表现相当。此外,PRISM在挑战性公开基准上表现出优越的泛化能力,在超过500个标记的提示上,比基线模型高出7.4%。

英文摘要:

While modern text-to-image (T2I) models excel at generating images from intricate prompts, they struggle to capture the key details when the inputs are descriptive paragraphs. This limitation stems from the prevalence of concise captions that shape their training distributions. Existing methods attempt to bridge this gap by either fine-tuning T2I models on long prompts, which generalizes poorly to longer lengths; or by projecting the oversize inputs into normal-prompt space and compromising fidelity. We propose Prompt Refraction for Intricate Scene Modeling (PRISM), a compositional approach that enables pre-trained T2I models to process long sequence inputs. PRISM uses a lightweight module to extract constituent representations from the long prompts. The T2I model makes independent noise predictions for each component, and their outputs are merged into a single denoising step using energy-based conjunction. We evaluate PRISM across a wide range of model architectures, showing comparable performances to models fine-tuned on the same training data. Furthermore, PRISM demonstrates superior generalization, outperforming baseline models by 7.4% on prompts over 500 tokens in a challenging public benchmark.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑