arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2411.05993cs.CVcs.AIeess.IV
收藏

一种用于图像重建的模块化条件扩散框架

A Modular Conditional Diffusion Framework for Image Reconstruction

MTS AI

Magauiya Zhussip, Iaroslav Koshelev, Stamatis Lefkimmiatis

AI总结:

针对现有扩散概率模型用于图像复原时任务专属、训练成本高的问题,提出模块化DP-IR框架,仅需训练0.7M参数的小模块即可结合预训练模型优势,还可四倍降低采样计算量,在多项任务基准上感知质量更优且保真度具竞争力。

摘要:

扩散概率模型(Diffusion Probabilistic Models, DPMs)近年来被用于处理各类盲图像复原(image restoration, IR)任务,在感知质量方面展现出优异性能。然而,现有方案的任务专属特性,以及其训练相关的过高计算成本,使得这类模型难以应用于初始训练目标之外的不同IR任务,实用性不足且使用门槛高。这阻碍了它们的更广泛应用,尤其是对于那些无法获取强大计算资源和海量训练数据的群体而言。\n在本研究中,我们旨在解决上述问题,推动DPMs在实际IR相关应用中的成功落地。为此,我们提出了一种模块化扩散概率IR框架(DP-IR),该框架能够结合现有预训练的顶尖IR网络与生成式DPMs的性能优势,同时仅需额外训练一个与目标IR任务相关的相对较小的模块(0.7M参数)。此外,所提框架的架构支持一种采样策略,可将神经函数评估次数减少至少四倍且不会造成任何性能损失,同时还可与DDIM等现有加速技术结合使用。\n我们在burst JDD-SR、动态场景去模糊和超分辨率任务的四个基准上评估了我们的模型。我们的方法在感知质量方面优于现有方法,同时在保真度指标上保持了具有竞争力的性能。

英文摘要:

Diffusion Probabilistic Models (DPMs) have been recently utilized to deal with various blind image restoration (IR) tasks, where they have demonstrated outstanding performance in terms of perceptual quality. However, the task-specific nature of existing solutions and the excessive computational costs related to their training, make such models impractical and challenging to use for different IR tasks than those that were initially trained for. This hinders their wider adoption, especially by those who lack access to powerful computational resources and vast amount of training data. In this work we aim to address the above issues and enable the successful adoption of DPMs in practical IR-related applications. Towards this goal, we propose a modular diffusion probabilistic IR framework (DP-IR), which allows us to combine the performance benefits of existing pre-trained state-of-the-art IR networks and generative DPMs, while it requires only the additional training of a relatively small module (0.7M params) related to the particular IR task of interest. Moreover, the architecture of the proposed framework allows for a sampling strategy that leads to at least four times reduction of neural function evaluations without suffering any performance loss, while it can also be combined with existing acceleration techniques such as DDIM. We evaluate our model on four benchmarks for the tasks of burst JDD-SR, dynamic scene deblurring, and super-resolution. Our method outperforms existing approaches in terms of perceptual quality while it retains a competitive performance with respect to fidelity metrics.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑