arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2504.03471cs.CV
收藏

扩散 U-Net 中的动态重要性用于增强图像合成

Dynamic Importance in Diffusion U-Net for Enhanced Image Synthesis

CSSE, Shenzhen University(深圳大学计算机科学与工程学院)

Xi Wang, Ziqi He, Yang Zhou

AI总结:

本文针对扩散 U-Net 中注意力模块重要性随推理过程动态变化的问题,提出 Importance Probe 量化方法及自适应重加权调度,显著提升推理效率和图像生成质量。

摘要:

传统扩散模型通常采用 U-Net 架构。以往研究揭示了 U-Net 中注意力模块的作用,但忽略了它们在推理过程中重要性的动态演变,这阻碍了进一步利用这些模块来改进图像应用。在本研究中,我们首先从理论上证明,对 U-Net 内 Transformer 模块的输出进行重新加权是提升采样过程中信噪比的“免费午餐”。其次,我们提出了 Importance Probe,用于揭示和量化整个去噪过程中 Transformer 模块重要性的动态变化。最后,我们设计了一种基于自适应重要性的重新加权调度方案,专门针对特定的图像生成和编辑任务。实验结果表明,我们的方法显著提高了推理过程的效率,并增强了样本的美学质量与身份一致性。我们的方法可以无缝集成到任何基于 U-Net 的架构中。代码:https://github.com/Hytidel/UNetReweighting

英文摘要:

Traditional diffusion models typically employ a U-Net architecture. Previous studies have unveiled the roles of attention blocks in the U-Net. However, they overlook the dynamic evolution of their importance during the inference process, which hinders their further exploitation to improve image applications. In this study, we first theoretically proved that, re-weighting the outputs of the Transformer blocks within the U-Net is a "free lunch" for improving the signal-to-noise ratio during the sampling process. Next, we proposed Importance Probe to uncover and quantify the dynamic shifts in importance of the Transformer blocks throughout the denoising process. Finally, we design an adaptive importance-based re-weighting schedule tailored to specific image generation and editing tasks. Experimental results demonstrate that, our approach significantly improves the efficiency of the inference process, and enhances the aesthetic quality of the samples with identity consistency. Our method can be seamlessly integrated into any U-Net-based architecture. Code: https://github.com/Hytidel/UNetReweighting

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑