HAIR:基于超网络的一体化图像复原
HAIR: Hypernetworks-based All-in-One Image Restoration
AI总结:
本文提出基于超网络的即插即用一体化图像复原方法 HAIR,通过分类器和超选择网络根据输入图像动态生成参数,使模型自适应不同退化类型,在单任务和一体化设置下均显著提升性能,且所需参数更少。
摘要:
图像复原旨在从退化版本中恢复高质量的干净图像。近年来,一体化图像复原模型在同时处理多种未知退化方面展现出有效性。然而,现有方法通常使用相同的参数来处理不同类型退化的图像,迫使模型在不同任务之间平衡性能,从而限制了其在每个任务上的表现。为缓解这一问题,我们提出 HAIR,一种基于超网络的一体化图像复原即插即用方法,它根据输入图像生成参数,从而使模型能够动态适应特定退化。具体而言,HAIR 由两个主要组件组成,即分类器和超选择网络(HSN)。分类器是一个简单的图像分类网络,用于生成包含输入图像退化信息的全局信息向量(GIV);HSN 是一个简单的全连接神经网络,接收 GIV 并为相应模块输出参数。大量实验表明,HAIR 能够以即插即用的方式显著提升现有图像复原模型的性能,无论是在单任务还是一体化设置下。值得注意的是,我们提出的模型 Res-HAIR 将 HAIR 集成到著名的 Restormer 中,与当前最先进的方法相比,可以获得更优或相当的性能。此外,我们从理论上证明,在达到给定的足够小的误差时,与主流的基于嵌入的一体化方法相比,我们提出的 HAIR 需要更少的参数。代码可在 https://github.com/toummHus/HAIR 获取。
英文摘要:
Image restoration aims to recover a high-quality clean image from its degraded version. Recent progress in image restoration has demonstrated the effectiveness of All-in-One image restoration models in addressing various unknown degradations simultaneously. However, these existing methods typically utilize the same parameters to tackle images with different types of degradation, forcing the model to balance the performance between different tasks and limiting its performance on each task. To alleviate this issue, we propose HAIR, a Hypernetworks-based All-in-One Image Restoration plug-and-play method that generates parameters based on the input image and thus makes the model to adapt to specific degradation dynamically. Specifically, HAIR consists of two main components, i.e., Classifier and Hyper Selecting Net (HSN). The Classifier is a simple image classification network used to generate a Global Information Vector (GIV) that contains the degradation information of the input image, and the HSN is a simple fully-connected neural network that receives the GIV and outputs parameters for the corresponding modules. Extensive experiments demonstrate that HAIR can significantly improve the performance of existing image restoration models in a plug-and-play manner, both in single-task and All-in-One settings. Notably, our proposed model Res-HAIR, which integrates HAIR into the well-known Restormer, can obtain superior or comparable performance compared with current state-of-the-art methods. Moreover, we theoretically demonstrate that to achieve a given small enough error, our proposed HAIR requires fewer parameters in contrast to mainstream embedding-based All-in-One methods. The code is available at https://github.com/toummHus/HAIR.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。