Image restoration (IR) aims to recover clean images from degraded observations. Despite remarkable progress, most existing methods focus on a single degradation type, whereas real-world images often suffer from multiple coexisting degradations, such as rain, noise, and haze coexisting in a single image, which limits their practical effectiveness. In this paper, we propose an adaptive multi-degradation image restoration network that reconstructs images by leveraging decoupled representations of degradation ingredients to guide path selection. Specifically, we design a degradation ingredient decoupling block (DIDBlock) in the encoder to separate degradation ingredients statistically by integrating spatial and frequency domain information, enhancing the recognition of multiple degradation types and making their feature representations independent. In addition, we present fusion block (FBlock) to integrate degradation information across all levels using learnable matrices. In the decoder, we further introduce a task adaptation block (TABlock) that dynamically activates or fuses functional branches based on the multi-degradation representation, flexibly selecting optimal restoration paths under diverse degradation conditions. The resulting tightly integrated architecture, termed IMDNet, is extensively validated through experiments, showing superior performance on multi-degradation restoration while maintaining strong competitiveness on single-degradation tasks.
Tokyo Institute of Technology（东京工业大学）；Kadinche Corporation（Kadinche公司）；National Institute of Advanced Industrial Science and Technology（国立先进工业科学技术研究所）
Image restoration from a single image degradation type, such as blurring, hazing, random noise, and compression has been investigated for decades. However, image degradations in practice are often a mixture of several types of degradation. Such compositional degradations complicate restoration because they require the differentiation of different degradation types and levels. In this paper, we propose a convolutional neural network (CNN) model for estimating the degradation properties of a given degraded image. Furthermore, we introduce an image restoration CNN model that adopts the estimated …
Image degradation due to atmospheric turbulence is common while capturing images at long ranges. To mitigate the degradation due to turbulence which includes deformation and blur, we propose a generative single frame restoration algorithm which disentangles the blur and deformation due to turbulence and reconstructs a restored image. The disentanglement is achieved by decomposing the distortion due to turbulence into blur and deformation components using deblur generator and deformation correction generator respectively. Two paths of restoration are implemented to regularize the disentanglemen…
Under-display camera (UDC) is a novel technology that can make digital imaging experience in handheld devices seamless by providing large screen-to-body ratio. UDC images are severely degraded owing to their positioning under a display screen. This work addresses the restoration of images degraded as a result of UDC imaging. Two different networks are proposed for the restoration of images taken with two types of UDC technologies. The first method uses a pyramidal dilated convolution within a wavelet decomposed convolutional neural network for pentile-organic LED (P-OLED) based display system.…
Jilin University（吉林大学）；University of Trento（特伦托大学）；ETH Zurich（苏黎世联邦理工学院）
Constructing high-quality character image datasets is challenging because real-world images are often affected by image degradation. There are limitations when applying current image restoration methods to such real-world character images, since (i) the categories of noise in character images are different from those in general images; (ii) real-world character images usually contain more complex image degradation, e.g., mixed noise at different noise levels. To address these problems, we propose a real-world character restoration network (RCRN) to effectively restore degraded character images…
University of Science and Technology of China（中国科学技术大学）；Microsoft Research（微软研究院）；University of Rochester（罗切斯特大学）
An important scenario for image quality assessment (IQA) is to evaluate image restoration (IR) algorithms. The state-of-the-art approaches adopt a full-reference paradigm that compares restored images with their corresponding pristine-quality images. However, pristine-quality images are usually unavailable in blind image restoration tasks and real-world scenarios. In this paper, we propose a practical solution named degraded-reference IQA (DR-IQA), which exploits the inputs of IR models, degraded images, as references. Specifically, we extract reference information from degraded images by dist…
Harbin Institute of Technology（哈尔滨工业大学）；The Hong Kong Polytechnic University（香港理工大学）；Nanyang Technological University（南洋理工大学）；Alibaba Group（阿里巴巴集团）；Peng Cheng Lab（鹏城实验室）
How to design proper training pairs is critical for super-resolving real-world low-quality (LQ) images, which suffers from the difficulties in either acquiring paired ground-truth high-quality (HQ) images or synthesizing photo-realistic degraded LQ observations. Recent works mainly focus on modeling the degradation with handcrafted or estimated degradation parameters, which are however incapable to model complicated real-world degradation types, resulting in limited quality improvement. Notably, LQ face images, which may have the same degradation process as natural images, can be robustly rest…
Restore, Assess, Repeat: A Unified Framework for Iterative Image Restoration
Samsung AI Center Cambridge（三星剑桥人工智能中心）；National Taiwan University（国立台湾大学）；Technical University of Iasi（雅西科技大学）；Chang Gung University（长庚大学）；University of Wurzburg（维尔茨堡大学）；Queen Mary University of London（伦敦玛丽女王大学）
Image restoration is the task of recovering a clean image from a degraded version. In most cases, the degradation is spatially varying, and it requires the restoration network to both localize and restore the affected regions. In this paper, we present a new approach suitable for handling the image-specific and spatially-varying nature of degradation in images affected by practically occurring artifacts such as blur, rain-streaks. We decompose the restoration task into two stages of degradation localization and degraded region-guided restoration, unlike existing methods which directly learn a …
Diff-Restorer: Unleashing Visual Prompts for Diffusion-based Universal Image Restoration
Institute of Image Communication and Network Engineering, Shanghai Jiao Tong University（上海交通大学图像通信与网络工程研究所）；MoE Key Lab of Artificial Intelligence, AI Institute, Shanghai Jiao Tong University（上海交通大学人工智能研究院、教育部人工智能重点实验室）
DACESR: Degradation-Aware Conditional Embedding for Real-World Image Super-Resolution
School of Electrical and Information Engineering, Zhengzhou University of Light Industry（郑州轻工业大学电气与信息工程学院）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；School of Automation, Central South University（中南大学自动化学院）；Intelligent Manufacturing Engineering at Machinery Technology Development Co.,Ltd.（机械技术发展有限公司智能制造工程）；China Academy of Machinery Science and Technology Group Co.,Ltd.（中国机械科学与技术集团有限公司）
Underwater visual perception is essentially important for underwater exploration, archeology, ecosystem and so on. The low illumination, light reflections, scattering, absorption and suspended particles inevitably lead to the critically degraded underwater image quality, which causes great challenges on recognizing the objects from the underwater images. The existing underwater enhancement methods that aim to promote the underwater visibility, heavily suffer from the poor image restoration performance and generalization ability. To reduce the difficulty of underwater image enhancement, we intr…
Imaging under photon-scarce situations introduces challenges to many applications as the captured images are with low signal-to-noise ratio and poor luminance. In this paper, we investigate the raw image restoration under low-photon-count conditions by simulating the imaging of quanta image sensor (QIS). We develop a lightweight framework, which consists of a multi-level pyramid denoising network (MPDNet) and a luminance adjustment (LA) module to achieve separate denoising and luminance enhancement. The main component of our framework is the multi-skip attention residual block (MARB), which in…
Whiting School of Engineering, Johns Hopkins University（约翰斯·霍普金斯大学惠廷工程学院）
Atmospheric turbulence significantly affects imaging systems which use light that has propagated through long atmospheric paths. Images captured under such condition suffer from a combination of geometric deformation and space varying blur. We present a deep learning-based solution to the problem of restoring a turbulence-degraded face image where prior information regarding the amount of geometric distortion and blur at each location of the face image is first estimated using two separate networks. The estimated prior information is then used by a network called, Turbulence Distortion Removal…
Perceive, Understand and Restore: Real-World Image Super-Resolution with Autoregressive Multimodal Generative Models
Tsinghua Shenzhen International Graduate School, Tsinghua University（清华大学深圳国际研究生院）；The Hong Kong Polytechnic University（香港理工大学）；OPPO Research Institute（OPPO研究院）
Atmospheric turbulence can significantly degrade the quality of images acquired by long-range imaging systems by causing spatially and temporally random fluctuations in the index of refraction of the atmosphere. Variations in the refractive index causes the captured images to be geometrically distorted and blurry. Hence, it is important to compensate for the visual degradation in images caused by atmospheric turbulence. In this paper, we propose a deep learning-based approach for restring a single image degraded by atmospheric turbulence. We make use of the epistemic uncertainty based on Monte…
Almost all existing methods for image restoration are based on optimizing the mean squared error (MSE), even though it is known that the best estimate in terms of MSE may yield a highly atypical image due to the fact that there are many plausible restorations for a given noisy image. In this paper, we show how to combine explicit priors on patches of natural images in order to sample from the posterior probability of a full image given a degraded image. We prove that our algorithm generates correct samples from the distribution $p(x|y) \propto \exp(-E(x|y))$ where $E(x|y)$ is the cost function…
From Zero to Detail: A Progressive Spectral Decoupling Paradigm for UHD Image Restoration with New Benchmark
State Key Laboratory of Novel Software Technology, Nanjing University（新型软件技术国家重点实验室，南京大学）；Department of Computer Science and Engineering, Michigan State University（计算机科学与工程系，密歇根州立大学）；School of Computer Science and Technology, Dalian University of Technology（计算机科学与技术学院，大连理工大学）
Super-Resolution in The Right Latent Space: A Frozen Vision-Foundation Substrate
University of California, Berkeley（加州大学伯克利分校）；Alibaba Group（阿里巴巴集团）；University of Michigan, Ann Arbor（密歇根大学安娜堡分校）；Nanjing University（南京大学）；Southeast University（东南大学）
Perceptual image restoration seeks for high-fidelity images that most likely degrade to given images. For better visual quality, previous work proposed to search for solutions within the natural image manifold, by exploiting the latent space of a generative model. However, the quality of generated images are only guaranteed when latent embedding lies close to the prior distribution. In this work, we propose to restrict the feasible region within the prior manifold. This is accomplished with a non-parametric metric for two distributions: the Maximum Mean Discrepancy (MMD). Moreover, we model th…
Image restoration tasks have achieved tremendous performance improvements with the rapid advancement of deep neural networks. However, most prevalent deep learning models perform inference statically, ignoring that different images have varying restoration difficulties and lightly degraded images can be well restored by slimmer subnetworks. To this end, we propose a new solution pipeline dubbed ClassPruning that utilizes networks with different capabilities to process images with varying restoration difficulties. In particular, we use a lightweight classifier to identify the image restoration …
Michigan State University（密歇根州立大学）；Indian Institute of Technology Madras（印度理工学院马德拉斯分校）
We present a general learning-based solution for restoring images suffering from spatially-varying degradations. Prior approaches are typically degradation-specific and employ the same processing across different images and different pixels within. However, we hypothesize that such spatially rigid processing is suboptimal for simultaneously restoring the degraded pixels as well as reconstructing the clean regions of the image. To overcome this limitation, we propose SPAIR, a network design that harnesses distortion-localization information and dynamically adjusts computation to difficult regio…
Universidade Federal do Rio Grande（里奥格兰德联邦大学）；Computer Vision Center（计算机视觉中心）
This paper describes a method to restore degraded images captured in a participating media -- fog, turbid water, sand storm, etc. Differently from the related work that only deal with a medium, we obtain generality by using an image formation model and a fusion of new image priors. The model considers the image color variation produced by the medium. The proposed restoration method is based on the fusion of these priors and supported by statistics collected on images acquired in both non-participating and participating media. The key of the method is to fuse two complementary measures --- loca…
Shanghai Jiao Tong University（上海交通大学）；University of Texas at Arlington（德克萨斯大学阿灵顿分校）；RIKEN AIP（理化学研究所革新智能综合研究中心）；The University of Tokyo（东京大学）；Innopeak Technology；The University of British Columbia（不列颠哥伦比亚大学）；Shanghai AI Laboratory（上海人工智能实验室）
Image restoration algorithms such as super resolution (SR) are indispensable pre-processing modules for object detection in degraded images. However, most of these algorithms assume the degradation is fixed and known a priori. When the real degradation is unknown or differs from assumption, both the pre-processing module and the consequent high-level task such as object detection would fail. Here, we propose a novel framework, RestoreDet, to detect objects in degraded low resolution images. RestoreDet utilizes the downsampling degradation as a kind of transformation for self-supervised signals…
Tsinghua University（清华大学）；Shenzhen Institutes of Advanced Technology (SIAT), Chinese Academy of Science（中国科学院深圳先进技术研究院）；Tsinghua Shenzhen International Graduate School（清华大学深圳国际研究生院）；University College London（伦敦大学学院）
Most state-of-the-art semantic segmentation approaches only achieve high accuracy in good conditions. In practically-common but less-discussed adverse environmental conditions, their performance can decrease enormously. Existing studies usually cast the handling of segmentation in adverse conditions as a separate post-processing step after signal restoration, making the segmentation performance largely depend on the quality of restoration. In this paper, we propose a novel deep-learning framework to tackle semantic segmentation and image restoration in adverse environmental conditions in a hol…
University of Electronic Science and Technology of China（电子科技大学）；Kuaishou（快手）
In this paper, we propose a luminance-guided chrominance image enhancement convolutional neural network for HEVC intra coding. Specifically, we firstly develop a gated recursive asymmetric-convolution block to restore each degraded chrominance image, which generates an intermediate output. Then, guided by the luminance image, the quality of this intermediate output is further improved, which finally produces the high-quality chrominance image. When our proposed method is adopted in the compression of color images with HEVC intra coding, it achieves 28.96% and 16.74% BD-rate gains over HEVC for…
Degradation Frequency Curve: An Explicit Frequency-Quantified Representation for All-in-One Image Restoration
College of Electronic Science, National University of Defense Technology（国防科技大学电子科学学院）；College of Electronic Engineering, Beihang University（北航电子工程学院）；VCIP, School of Computer Science, Nankai University（南开大学计算机科学学院）
When Color-Space Decoupling Meets Diffusion for Adverse-Weather Image Restoration
School of Computer Science and Engineering, Nanjing University of Science and Technology（计算机科学与工程学院，南京理工大学）；School of Electrical Engineering, Southeast University（电气工程学院，东南大学）；School of Intelligence Science and Technology, Nanjing University（智能科学与技术学院，南京大学）
Tianjin University（天津大学）；Hong Kong Polytechnic University（香港理工大学）；Harbin Institute of Technology（哈尔滨工业大学）；Chinese University of Hong Kong (Shenzhen)（香港中文大学（深圳））；University of California, Merced（加州大学默塞德分校）
Most existing non-blind restoration methods are based on the assumption that a precise degradation model is known. As the degradation process can only be partially known or inaccurately modeled, images may not be well restored. Rain streak removal and image deconvolution with inaccurate blur kernels are two representative examples of such tasks. For rain streak removal, although an input image can be decomposed into a scene layer and a rain streak layer, there exists no explicit formulation for modeling rain streaks and the composition with scene layer. For blind deconvolution, as estimation e…
