arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2409.20197cs.CV

UIR-LoRA:通过多重低秩适配实现通用图像恢复

UIR-LoRA: Achieving Universal Image Restoration through Multiple Low-Rank Adaptation

Northwestern Polytechnical University(西北工业大学);University of New South Wales(新南威尔士大学)

Cheng Zhang, Dong Gong, Jiumei He, Yu Zhu, Jinqiu Sun, Yanning Zhang

AI总结:

本文提出UIR-LoRA框架,利用预训练生成模型与多重低秩适配器,通过基于退化相似性的LoRA组合策略,实现多种及混合退化的通用图像恢复,在保真度、感知质量和泛化能力上优于现有统一模型。

摘要:

现有的统一方法通常将多退化图像恢复视为多任务学习问题。尽管与单退化恢复方法相比表现有效,但它们忽视了多任务恢复中共同性与特异性的利用,从而阻碍了模型性能。受深度生成模型和微调技术成功的启发,我们提出了一种基于多域迁移学习中多重低秩适配器(LoRA)的通用图像恢复框架。我们的框架利用预训练生成模型作为多退化恢复的共享组件,并通过低秩适配将其迁移到特定退化图像恢复任务中。此外,我们引入了一种基于退化相似性的LoRA组合策略,该策略自适应地组合已训练的LoRA,使我们的模型能够适用于混合退化恢复。在多种和混合退化上的大量实验表明,所提出的通用图像恢复方法不仅实现了更高的保真度和感知图像质量,而且比其他统一图像恢复模型具有更好的泛化能力。我们的代码可在https://github.com/Justones/UIR-LoRA获取。

英文摘要:

Existing unified methods typically treat multi-degradation image restoration as a multi-task learning problem. Despite performing effectively compared to single degradation restoration methods, they overlook the utilization of commonalities and specificities within multi-task restoration, thereby impeding the model's performance. Inspired by the success of deep generative models and fine-tuning techniques, we proposed a universal image restoration framework based on multiple low-rank adapters (LoRA) from multi-domain transfer learning. Our framework leverages the pre-trained generative model as the shared component for multi-degradation restoration and transfers it to specific degradation image restoration tasks using low-rank adaptation. Additionally, we introduce a LoRA composing strategy based on the degradation similarity, which adaptively combines trained LoRAs and enables our model to be applicable for mixed degradation restoration. Extensive experiments on multiple and mixed degradations demonstrate that the proposed universal image restoration method not only achieves higher fidelity and perceptual image quality but also has better generalization ability than other unified image restoration models. Our code is available at https://github.com/Justones/UIR-LoRA.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑