arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2410.01540cs.CVcs.AIcs.GRcs.LG
收藏

边缘保留噪声用于扩散模型

Edge-preserving noise for diffusion models

Max Planck Institute for Informatics(马克斯·普朗克信息研究所);Advanced Micro Devices(先进微器件)

Jente Vandersanden, Sascha Holl, Xingchang Huang, Gurprit Singh

AI总结:

本文提出一种边缘保留的扩散过程,通过混合噪声方案在边缘感知调度器中平滑过渡,提升结构细节捕捉能力,同时保持全局性能,并在图像生成和结构引导任务中取得改进。

摘要:

经典扩散模型通常依赖各向同性高斯噪声,对所有区域一视同仁,忽略了对高质量生成重要的结构信息。我们引入了一种边缘保留的扩散过程,通过混合噪声方案扩展各向同性模型,采用边缘感知调度器,平滑过渡从边缘保留到各向同性噪声。这使模型能够捕捉精细的结构细节,同时在总体上保持良好的性能。我们评估了结构感知噪声在扩散和流匹配框架中的影响,并展示现有各向同性模型可以高效地通过边缘保留噪声进行微调,使我们的框架适用于适应预训练系统。除了无条件生成外,我们的方法在结构引导任务如笔触到图像合成中显示出改进,通过FID、KID和CLIP-score的一致改进得到验证。

英文摘要:

Classical diffusion models typically rely on isotropic Gaussian noise, treating all regions uniformly and overlooking structural information important for high-quality generation. We introduce an edge-preserving diffusion process that generalizes isotropic models via a hybrid noise scheme with an edge-aware scheduler that smoothly transitions from edge-preserving to isotropic noise. This enables the model to capture fine structural details while generally maintaining global performance. We evaluate the impact of structure-aware noise in both diffusion and flow-matching frameworks, and show that existing isotropic models can be efficiently fine-tuned with edge-preserving noise, making our framework practical for adapting pre-trained systems. Beyond unconditional generation, our method particularly shows improvements in structure-guided tasks such as stroke-to-image synthesis, improving robustness and perceptual quality, as evidenced by consistent improvements across FID, KID, and CLIP-score.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑