NoiseCollage：一种基于噪声裁剪与合并的布局感知文本生成图像扩散模型
NoiseCollage: A Layout-Aware Text-to-Image Diffusion Model Based on Noise Cropping and Merging
AI总结：
针对布局感知文本生成图像扩散模型存在的文图条件不匹配、图像质量下降问题，提出基于噪声裁剪合并的NoiseCollage模型，性能优于现有最优模型，还可集成ControlNet提升其布局精度。
摘要：
布局感知文本生成图像是指除文本条件外，还需生成符合布局条件的多目标图像的任务。当前布局感知文本生成图像扩散模型仍存在若干问题，包括文本与布局条件不匹配、生成图像质量下降等。本文提出了一种名为NoiseCollage的新型布局感知文本生成图像扩散模型以解决这些问题。在去噪过程中，NoiseCollage为单个目标独立估计噪声，随后将这些噪声裁剪并合并为单一噪声。该操作有助于避免条件不匹配，换言之，它能将正确的目标放置在正确的位置。定性与定量评估表明，NoiseCollage的性能优于多个当前最优模型。这些成功的结果表明，噪声的裁剪与合并操作是控制图像生成的一种合理策略。我们还展示了NoiseCollage可与ControlNet集成，以将边缘、草图和姿态骨架作为附加条件。实验结果表明，这种集成提升了ControlNet的布局精度。代码可在https://github.com/univ-esuty/noisecollage获取。
英文摘要：
Layout-aware text-to-image generation is a task to generate multi-object images that reflect layout conditions in addition to text conditions. The current layout-aware text-to-image diffusion models still have several issues, including mismatches between the text and layout conditions and quality degradation of generated images. This paper proposes a novel layout-aware text-to-image diffusion model called NoiseCollage to tackle these issues. During the denoising process, NoiseCollage independently estimates noises for individual objects and then crops and merges them into a single noise. This operation helps avoid condition mismatches; in other words, it can put the right objects in the right places. Qualitative and quantitative evaluations show that NoiseCollage outperforms several state-of-the-art models. These successful results indicate that the crop-and-merge operation of noises is a reasonable strategy to control image generation. We also show that NoiseCollage can be integrated with ControlNet to use edges, sketches, and pose skeletons as additional conditions. Experimental results show that this integration boosts the layout accuracy of ControlNet. The code is available at https://github.com/univ-esuty/noisecollage.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
