Image restoration is a long-standing low-level vision problem, e.g., deblurring and deraining. In the process of image restoration, it is necessary to consider not only the spatial details and contextual information of restoration to ensure the quality, but also the system complexity. Although many methods have been able to guarantee the quality of image restoration, the system complexity of the state-of-the-art (SOTA) methods is increasing as well. Motivated by this, we present a mixed hierarchy network that can balance these competing goals. Our main proposal is a mixed hierarchy architecture, that progressively recovers contextual information and spatial details from degraded images while we design intra-blocks to reduce system complexity. Specifically, our model first learns the contextual information using encoder-decoder architectures, and then combines them with high-resolution branches that preserve spatial detail. In order to reduce the system complexity of this architecture for convenient analysis and comparison, we replace or remove the nonlinear activation function with multiplication and use a simple network structure. In addition, we replace spatial convolution with global self-attention for the middle block of encoder-decoder. The resulting tightly interlinked hierarchy architecture, named as MHNet, delivers strong performance gains on several image restoration tasks, including image deraining, and deblurring.
An Efficient and Mixed Heterogeneous Model for Image Restoration
MAC Lab, Xiamen University, China（厦门大学）；National University of Singapore, Singapore（新加坡国立大学）；Smart City Research Institute, China Electronics Technology Group Corporation, China（中国电子科技集团有限公司智慧城市研究院）
This article describes a fast iterative algorithm for image denoising and deconvolution with signal-dependent observation noise. We use an optimization strategy based on variable splitting that adapts traditional Gaussian noise-based restoration algorithms to account for the observed image being corrupted by mixed Poisson-Gaussian noise and quantization errors.
Chinese University of Hongkong（香港中文大学）；University of Electronic Science and Technology of China（电子科技大学）
Hyperspectral images (HSIs) are often corrupted by a mixture of several types of noise during the acquisition process, e.g., Gaussian noise, impulse noise, dead lines, stripes, and many others. Such complex noise could degrade the quality of the acquired HSIs, limiting the precision of the subsequent processing. In this paper, we present a novel tensor-based HSI restoration approach by fully identifying the intrinsic structures of the clean HSI part and the mixed noise part respectively. Specifically, for the clean HSI part, we use tensor Tucker decomposition to describe the global correlation…
ClusIR: Towards Cluster-Guided All-in-One Image Restoration
Zhongnan University of Economics and Law（中南财经政法大学）；Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）；Yunnan University（云南大学）；Nanyang Technological University（南洋理工大学）；Wuhan University（武汉大学）
Jilin University（吉林大学）；University of Trento（特伦托大学）；ETH Zurich（苏黎世联邦理工学院）
Constructing high-quality character image datasets is challenging because real-world images are often affected by image degradation. There are limitations when applying current image restoration methods to such real-world character images, since (i) the categories of noise in character images are different from those in general images; (ii) real-world character images usually contain more complex image degradation, e.g., mixed noise at different noise levels. To address these problems, we propose a real-world character restoration network (RCRN) to effectively restore degraded character images…
Visual-Instructed Degradation Diffusion for All-in-One Image Restoration
State Key Laboratory of Multimodal Artificial Intelligence Systems (MAIS), CASIA（多模态人工智能系统国家重点实验室（MAIS）、中国科学院自动化研究所）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）；Ant Group（蚂蚁集团）；PeopleAI Inc.（PeopleAI公司）
Northwest A\&F University, College of Science, Yangling 712100, P.R. China
Several bandwise total variation (TV) regularized low-rank (LR)-based models have been proposed to remove mixed noise in hyperspectral images (HSIs). Conventionally, the rank of LR matrix is approximated using nuclear norm (NN). The NN is defined by adding all singular values together, which is essentially a $L_1$-norm of the singular values. It results in non-negligible approximation errors and thus the resulting matrix estimator can be significantly biased. Moreover, these bandwise TV-based methods exploit the spatial information in a separate manner. To cope with these problems, we propose …
DPMambaIR: All-in-One Image Restoration via Degradation-Aware Prompt State Space Model
School of Information Engineering, Chang’an University（信息工程学院，长安大学）；School of Electronics and Information, Northwestern Polytechnical University（电子与信息学院，西北工业大学）
Universal Image Restoration via Internalized Chain-of-Thought Reasoning
Hong Kong JC Lab of Smart City and Department of Computer Science, City University of Hong Kong（香港城市大学智慧城市香港联合实验室及计算机科学系）；School of Computing and Information Systems, Singapore Management University（新加坡管理大学计算与信息系统学院）；Computer Science and Information Engineering, National Taiwan University（国立台湾大学计算机科学与信息工程学系）
In this paper, we propose a residual non-local attention network for high-quality image restoration. Without considering the uneven distribution of information in the corrupted images, previous methods are restricted by local convolutional operation and equal treatment of spatial- and channel-wise features. To address this issue, we design local and non-local attention blocks to extract features that capture the long-range dependencies between pixels and pay more attention to the challenging parts. Specifically, we design trunk branch and (non-)local mask branch in each (non-)local attention b…
Degradation Frequency Curve: An Explicit Frequency-Quantified Representation for All-in-One Image Restoration
College of Electronic Science, National University of Defense Technology（国防科技大学电子科学学院）；College of Electronic Engineering, Beihang University（北航电子工程学院）；VCIP, School of Computer Science, Nankai University（南开大学计算机科学学院）
The Chinese University of Hong Kong（香港中文大学）；Tencent PCG（腾讯平台与内容事业群）；Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）；Nanyang Technological University（南洋理工大学）
Very deep Convolutional Neural Networks (CNNs) have greatly improved the performance on various image restoration tasks. However, this comes at a price of increasing computational burden, hence limiting their practical usages. We observe that some corrupted image regions are inherently easier to restore than others since the distortion and content vary within an image. To leverage this, we propose Path-Restore, a multi-path CNN with a pathfinder that can dynamically select an appropriate route for each image region. We train the pathfinder using reinforcement learning with a difficulty-regulat…
Universal Image Restoration Pre-training via Degradation Classification
Institute of Medical Technology, Peking University Health Science Center, Peking University（北京大学医学部医学技术研究院）；College of Future Technology, Peking University（北京大学未来技术学院）；National Biomedical Imaging Center, Peking University（北京大学生物医学成像中心）；China Mobile Research Institute（中国移动研究院）
UniUIR: Considering Underwater Image Restoration as An All-in-One Learner
National Engineering Research Center for Multimedia Software, School of Computer Science, Wuhan University（国家多媒体软件工程研究中心，计算机学院，武汉大学）；School of Information Engineering, Guangdong University of Technology（信息工程学院，广东工业大学）；Horizon Robotics
National University of Singapore（新加坡国立大学）；Yale-NUS College（耶鲁-新加坡国立大学学院）
Most deraining works focus on rain streaks removal but they cannot deal adequately with heavy rain images. In heavy rain, streaks are strongly visible, dense rain accumulation or rain veiling effect significantly washes out the image, further scenes are relatively more blurry, etc. In this paper, we propose a novel method to address these problems. We put forth a 2-stage network: a physics-based backbone followed by a depth-guided GAN refinement. The first stage estimates the rain streaks, the transmission, and the atmospheric light governed by the underlying physics. To tease out these compon…
PaAgent: Portrait-Aware Image Restoration Agent via Subjective-Objective Reinforcement Learning
School of Computer Science, Northwestern Polytechnical University（西北工业大学计算机学院）；Shenzhen Research Institute of Northwestern Polytechnical University（西北工业大学深圳研究院）；State Key Laboratory of Internet of Things for Smart City, University of Macau（澳门大学智慧城市物联网国家重点实验室）；National-Local Joint Engineering Research Center of Biodiagnosis and Biotherapy, the Second Affiliated Hospital of Xi’an Jiaotong University（西安交通大学生物诊断与生物治疗国家地方联合工程研究中心）；College of Information and Control Engineering, Xi’an University of Architecture and Technology（西安建筑科技大学信息与控制工程学院）
Harbin Institute of Technology（哈尔滨工业大学）；Peking University（北京大学）
This paper presents a novel strategy for high-fidelity image restoration by characterizing both local smoothness and nonlocal self-similarity of natural images in a unified statistical manner. The main contributions are three-folds. First, from the perspective of image statistics, a joint statistical modeling (JSM) in an adaptive hybrid space-transform domain is established, which offers a powerful mechanism of combining local smoothness and nonlocal self-similarity simultaneously to ensure a more reliable and robust estimation. Second, a new form of minimization functional for solving image i…
ContiLNN: Mitigating Slice Sampling Discontinuity with Liquid Neural Networks for Medical Image Restoration
Faculty of Information Engineering and Automation, Kunming University of Science and Technology（昆明理工大学信息工程与自动化学院）；Academy of Artificial Intelligence and Advanced Technology, Xi’an Jiaotong-Liverpool University（西交利物浦大学人工智能与先进技术研究院）；Massachusetts General Hospital（麻省总医院）；Harvard Medical School（哈佛医学院）
The motivation of this paper is to introduce a novel framework for the restoration of images corrupted by mixed Gaussian-impulse noise. To this aim, first, an adaptive curvelet thresholding criterion is proposed which tries to adaptively remove the perturbations appeared during denoising process. Then, a new statistical regularization term, called joint adaptive statistical prior (JASP), is established which enforces both the local and nonlocal statistical consistencies, simultaneously, in a unified manner. Furthermore, a novel technique for mixed Gaussian plus impulse noise removal using JASP…
Clear Roads, Clear Vision: Advancements in Multi-Weather Restoration for Smart Transportation
Finolex Academy of Management and Technology（Finolex管理与技术学院）；Drone Lab, Centre for Artificial Intelligence and Robotics, IIT Mandi（印度理工学院曼迪分校人工智能与机器人中心无人机实验室）；Mehta Family School of Data Science and Artificial Intelligence, IIT Guwahati（印度理工学院古瓦哈提分校梅塔家族数据科学与人工智能学院）；SPACE42；Centre of Excellence: Artificial Intelligence, School of Computer Science, UPES Dehradun（UPES德拉敦分校计算机科学学院人工智能卓越中心）
