arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2412.03355cs.CV

TASR:用于图像超分辨率的时间步感知扩散模型

TASR: Timestep-Aware Diffusion Model for Image Super-Resolution

Shenzhen International Graduate School, Tsinghua University(清华大学深圳国际研究生院);Meituan Inc.(美团公司)

Qinwei Lin, Xiaopeng Sun, Yu Gao, Yujie Zhong, Dengjie Li, Zheng Zhao, Haoqian Wang

AI总结:

提出时间步感知扩散模型TASR,根据去噪阶段动态融合ControlNet与SD特征,早期保真后期增强细节,提升图像超分辨率效果。

摘要:

扩散模型近年来在图像超分辨率领域取得了显著成果。这些方法通常通过ControlNet注入低分辨率(LR)图像。本文首先探索了通过ControlNet进行信息注入的时间动态,揭示出LR图像的输入主要影响去噪过程的初始阶段。基于这一发现,我们提出了一种新颖的时间步感知扩散模型,该模型自适应地整合来自ControlNet和预训练的Stable Diffusion(SD)的特征。我们的方法在扩散早期增强LR信息的传递以保证图像保真度,并在后期更多地激发SD模型自身的生成能力以增强生成图像的细节。为训练该方法,我们提出了一种时间步感知的训练策略,该策略在不同时间步采用不同的损失,并作用于不同的模块。在基准数据集上的实验证明了我们方法的有效性。代码:https://github.com/SleepyLin/TASR

英文摘要:

Diffusion models have recently achieved outstanding results in the field of image super-resolution. These methods typically inject low-resolution (LR) images via ControlNet.In this paper, we first explore the temporal dynamics of information infusion through ControlNet, revealing that the input from LR images predominantly influences the initial stages of the denoising process. Leveraging this insight, we introduce a novel timestep-aware diffusion model that adaptively integrates features from both ControlNet and the pre-trained Stable Diffusion (SD). Our method enhances the transmission of LR information in the early stages of diffusion to guarantee image fidelity and stimulates the generation ability of the SD model itself more in the later stages to enhance the detail of generated images. To train this method, we propose a timestep-aware training strategy that adopts distinct losses at varying timesteps and acts on disparate modules. Experiments on benchmark datasets demonstrate the effectiveness of our method. Code: https://github.com/SleepyLin/TASR

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑