TAUE:无需训练的噪声移植与培育扩散模型
TAUE: Training-free Noise Transplant and Cultivation Diffusion Model
AI总结:
TAUE提出无需训练的噪声移植与培育扩散模型,通过嵌入全局结构信息和语义线索,实现多层图像生成,提升跨层一致性并支持新应用。
摘要:
尽管文本到图像扩散模型取得显著成功,但其输出单一扁平图像仍限制了需要逐层控制的专业应用。现有方法或依赖大规模不可用数据集微调,或为训练自由但仅能生成孤立前景元素。为此,我们引入无需训练的噪声移植与培育扩散模型(TAUE),一种新的多层图像生成框架,无需微调或额外数据。TAUE将中间去噪潜在的全局结构信息嵌入初始噪声以保持空间一致性,并通过跨层注意力共享整合语义线索以维持上下文和视觉一致性。广泛实验表明,TAUE在无训练方法中取得最佳性能,生成图像质量与微调模型相当,同时提高跨层一致性。此外,它支持新应用,如布局感知编辑、多物体合成和背景替换,表明在现实创意工作流中具有交互式、分层生成系统的潜力。
英文摘要:
Despite the remarkable success of text-to-image diffusion models, their output of a single, flattened image remains a critical bottleneck for professional applications requiring layer-wise control. Existing solutions either rely on fine-tuning with large, inaccessible datasets or are training-free yet limited to generating isolated foreground elements, failing to produce a complete and coherent scene. To address this, we introduce the Training-free Noise Transplantation and Cultivation Diffusion Model (TAUE), a novel framework for layer-wise image generation that requires neither fine-tuning nor additional data. TAUE embeds global structural information from intermediate denoising latents into the initial noise to preserve spatial coherence, and integrates semantic cues through cross-layer attention sharing to maintain contextual and visual consistency across layers. Extensive experiments demonstrate that TAUE achieves state-of-the-art performance among training-free methods, delivering image quality comparable to fine-tuned models while improving inter-layer consistency. Moreover, it enables new applications, such as layout-aware editing, multi-object composition, and background replacement, indicating potential for interactive, layer-separated generation systems in real-world creative workflows.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。