Inception Institute of AI（Inception人工智能研究所）；Mohamed bin Zayed University of AI（穆罕默德·本·扎耶德人工智能大学）；Monash University（莫纳什大学）；University of California, Merced（加州大学默塞德分校）；Yonsei University（延世大学）；Google Research（谷歌研究院）
Syed Waqas Zamir, Aditya Arora, Salman Khan, Munawar Hayat, Fahad Shahbaz Khan, Ming-Hsuan Yang, Ling Shao
Image restoration tasks demand a complex balance between spatial details and high-level contextualized information while recovering images. In this paper, we propose a novel synergistic design that can optimally balance these competing goals. Our main proposal is a multi-stage architecture, that progressively learns restoration functions for the degraded inputs, thereby breaking down the overall recovery process into more manageable steps. Specifically, our model first learns the contextualized features using encoder-decoder architectures and later combines them with a high-resolution branch that retains local information. At each stage, we introduce a novel per-pixel adaptive design that leverages in-situ supervised attention to reweight the local features. A key ingredient in such a multi-stage architecture is the information exchange between different stages. To this end, we propose a two-faceted approach where the information is not only exchanged sequentially from early to late stages, but lateral connections between feature processing blocks also exist to avoid any loss of information. The resulting tightly interlinked multi-stage architecture, named as MPRNet, delivers strong performance gains on ten datasets across a range of tasks including image deraining, deblurring, and denoising. The source code and pre-trained models are available at https://github.com/swz30/MPRNet.
From Zero to Detail: A Progressive Spectral Decoupling Paradigm for UHD Image Restoration with New Benchmark
State Key Laboratory of Novel Software Technology, Nanjing University（新型软件技术国家重点实验室，南京大学）；Department of Computer Science and Engineering, Michigan State University（计算机科学与工程系，密歇根州立大学）；School of Computer Science and Technology, Dalian University of Technology（计算机科学与技术学院，大连理工大学）
Progressive Image Restoration via Text-Conditioned Video Generation
Department of Computer Science, University of Illinois Springfield（伊利诺伊大学斯普林菲尔德分校计算机科学系）；School of Electrical and Computer Engineering, Purdue University（普渡大学电气与计算机工程学院）
HDR is an important part of computational photography technology. In this paper, we propose a lightweight neural network called Efficient Attention-and-alignment-guided Progressive Network (EAPNet) for the challenge NTIRE 2022 HDR Track 1 and Track 2. We introduce a multi-dimensional lightweight encoding module to extract features. Besides, we propose Progressive Dilated U-shape Block (PDUB) that can be a progressive plug-and-play module for dynamically tuning MAccs and PSNR. Finally, we use fast and low-power feature-align module to deal with misalignment problem in place of the time-consumin…
As a widely studied task, video restoration aims to enhance the quality of the videos with multiple potential degradations, such as noises, blurs and compression artifacts. Among video restorations, compressed video quality enhancement and video super-resolution are two of the main tacks with significant values in practical scenarios. Recently, recurrent neural networks and transformers attract increasing research interests in this field, due to their impressive capability in sequence-to-sequence modeling. However, the training of these models is not only costly but also relatively hard to con…
Hong Kong Baptist University（香港浸会大学）；Institute of High Performance Computing, A*STAR（新加坡科技研究局高性能计算研究院）；vivo Mobile Communication Co. Ltd（维沃移动通信有限公司）；City University of Hong Kong（香港城市大学）；Nanyang Technological University（南洋理工大学）
High Dynamic Range (HDR) imaging via multi-exposure fusion is an important task for most modern imaging platforms. In spite of recent developments in both hardware and algorithm innovations, challenges remain over content association ambiguities caused by saturation, motion, and various artifacts introduced during multi-exposure fusion such as ghosting, noise, and blur. In this work, we propose an Attention-guided Progressive Neural Texture Fusion (APNT-Fusion) HDR restoration model which aims to address these issues within one framework. An efficient two-stream structure is proposed which sep…
University of Science and Technology of China（中国科学技术大学）
Image deblurring has seen a great improvement with the development of deep neural networks. In practice, however, blurry images often suffer from additional degradations such as downscaling and compression. To address these challenges, we propose an Enhanced Deep Pyramid Network (EDPN) for blurry image restoration from multiple degradations, by fully exploiting the self- and cross-scale similarities in the degraded image.Specifically, we design two pyramid-based modules, i.e., the pyramid progressive transfer (PPT) module and the pyramid self-attention (PSA) module, as the main components of t…
Tianjin University（天津大学）；College of Intelligence and Computing, Tianjin University（天津大学智能与计算学部）
The face super-resolution (FSR) task is to reconstruct high-resolution face images from low-resolution inputs. Recent works have achieved success on this task by utilizing facial priors such as facial landmarks. Most existing methods pay more attention to global shape and structure information, but less to local texture information, which makes them cannot recover local details well. In this paper, we propose a novel recurrent convolutional network based framework for face super-resolution, which progressively introduces both global shape and local texture information. We take full advantage o…
TIDE: Two-Stage Inverse Degradation Estimation with Guided Prior Disentanglement for Underwater Image Restoration
Mohamed bin Zayed University of AI（莫扎德·本·泽亚德人工智能大学）；University of Amsterdam（阿姆斯特丹大学）；University of Massachusetts, Amherst（马萨诸塞大学阿默斯特分校）；Nagasaki University（长崎大学）
A Progressive Image Restoration Network for High-order Degradation Imaging in Remote Sensing
School of Mathematics and Computational Science, Xiangtan University（湘潭大学数学与计算科学学院）；Hunan Key Laboratory for Computation and Simulation in Science and Engineering（湖南省计算与模拟科学工程重点实验室）；Key Laboratory for Intelligent Computing and Information Processing of the Ministry of Education（教育部智能计算与信息处理重点实验室）；National Center for Applied Mathematics in Hunan（湖南省应用数学中心）；Hunan International Scientific and Technological Innovation Cooperation Base of Computational Science（湖南省 computational science 国际科学和技术创新合作基地）；College of Mathematical Medicine, Zhejiang Normal University（浙江师范大学数学医学院）；School of Automation and Electronic Information, Xiangtan University（湘潭大学自动化与电子信息学院）
DiffBIR: Towards Blind Image Restoration with Generative Diffusion Prior
Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）；Shanghai AI Laboratory（上海人工智能实验室）；The Chinese University of Hong Kong（香港中文大学）
Infrared-Assisted Single-Stage Framework for Joint Restoration and Fusion of Visible and Infrared Images under Hazy Conditions
Faculty of Information Engineering and Automation（信息工程与自动化学院）；Key Laboratory of Artificial Intelligence in Yunnan Province（云南省人工智能重点实验室）；School of Instrument Science and Opto-Electronic Engineering（仪器科学与光电工程学院）
Distribution-aware Dataset Distillation for Efficient Image Restoration
Sun Yat‑sen University（中山大学）；Fuzhou University（福州大学）；University of Science and Technology of China（中国科学技术大学）；Nanjing University of Science and Technology（南京理工大学）
Perceive-IR: Learning to Perceive Degradation Better for All-in-One Image Restoration
National Engineering Research Center for Multimedia Software, School of Computer Science, Wuhan University（国家多媒体软件工程研究中心，计算机学院，武汉大学）；School of Computer Science, Wuhan University（计算机学院，武汉大学）；Horizon Robotics（_horizon robotics）；School of Information Engineering, Guangdong University of Technology（信息工程学院，广东技术大学）
The Chinese University of Hong Kong（香港中文大学）；Smartmore Corporation（思谋科技）
Long-range temporal alignment is critical yet challenging for video restoration tasks. Recently, some works attempt to divide the long-range alignment into several sub-alignments and handle them progressively. Although this operation is helpful in modeling distant correspondences, error accumulation is inevitable due to the propagation mechanism. In this work, we present a novel, generic iterative alignment module which employs a gradual refinement scheme for sub-alignments, yielding more accurate motion compensation. To further enhance the alignment accuracy and temporal consistency, we devel…
University of Science and Technology of China（中国科学技术大学）；Hefei Comprehensive National Science Center（合肥综合性国家科学中心）；Huawei Cloud BU（华为云业务单元）；Huawei Noah’s Ark Lab（华为诺亚方舟实验室）；University of Macau（澳门大学）；Shenzhen Institutes of Advanced Technology, CAS（中国科学院深圳先进技术研究院）
Learning a generalized prior for natural image restoration is an important yet challenging task. Early methods mostly involved handcrafted priors including normalized sparsity, l_0 gradients, dark channel priors, etc. Recently, deep neural networks have been used to learn various image priors but do not guarantee to generalize. In this paper, we propose a novel approach that embeds a task-agnostic prior into a transformer. Our approach, named Task-Agnostic Prior Embedding (TAPE), consists of two stages, namely, task-agnostic pre-training and task-specific fine-tuning, where the first stage emb…
University of Science and Technology of China（中国科学技术大学）
Deep learning provides a new avenue for image restoration, which demands a delicate balance between fine-grained details and high-level contextualized information during recovering the latent clear image. In practice, however, existing methods empirically construct encapsulated end-to-end mapping networks without deepening into the rationality, and neglect the intrinsic prior knowledge of restoration task. To solve the above problems, inspired by Taylor's Approximations, we unfold Taylor's Formula to construct a novel framework for image restoration. We find the main part and the derivative pa…
Deep LoRA-Unfolding Networks for Image Restoration
School of Computer Science and Technology, Harbin Institute of Technology (Shenzhen)（计算机科学与技术学院，哈尔滨工业大学（深圳））；Institute of Image Communication and Network Engineering, Shanghai Jiao Tong University（图像通信与网络工程院，上海交通大学）；School of Intelligence Science and Technology, Nanjing University（智能科学与技术学院，南京大学）；School of Electronics and Information Engineering, Harbin Institute of Technology (Shenzhen)（电子与信息工程学院，哈尔滨工业大学（深圳））
National University of Singapore（新加坡国立大学）；Yale-NUS College（耶鲁-新加坡国立大学学院）
Most deraining works focus on rain streaks removal but they cannot deal adequately with heavy rain images. In heavy rain, streaks are strongly visible, dense rain accumulation or rain veiling effect significantly washes out the image, further scenes are relatively more blurry, etc. In this paper, we propose a novel method to address these problems. We put forth a 2-stage network: a physics-based backbone followed by a depth-guided GAN refinement. The first stage estimates the rain streaks, the transmission, and the atmospheric light governed by the underlying physics. To tease out these compon…
University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；Bytedance Inc.（字节跳动公司）
While scale-invariant modeling has substantially boosted the performance of visual recognition tasks, it remains largely under-explored in deep networks based image restoration. Naively applying those scale-invariant techniques (e.g. multi-scale testing, random-scale data augmentation) to image restoration tasks usually leads to inferior performance. In this paper, we show that properly modeling scale-invariance into neural networks can bring significant benefits to image restoration performance. Inspired from spatial-wise convolution for shift-invariance, "scale-wise convolution" is proposed …
The Chinese University of Hong Kong（香港中文大学）；SenseTime Research（商汤科技研究院）；Sun Yat-sen University（中山大学）；CUHK - SenseTime Joint Lab（香港中文大学-商汤科技联合实验室）
We investigate a novel approach for image restoration by reinforcement learning. Unlike existing studies that mostly train a single large network for a specialized task, we prepare a toolbox consisting of small-scale convolutional networks of different complexities and specialized in different tasks. Our method, RL-Restore, then learns a policy to select appropriate tools from the toolbox to progressively restore the quality of a corrupted image. We formulate a step-wise reward function proportional to how well the image is restored at each step to learn the action policy. We also devise a joi…
TU Darmstadt（达姆施塔特工业大学）；TU Dresden（德累斯顿工业大学）；Microsoft Research Ltd（微软研究院）；Nuance Communications（纽安斯通信公司）
Conditional random fields (CRFs) are popular discriminative models for computer vision and have been successfully applied in the domain of image restoration, especially to image denoising. For image deblurring, however, discriminative approaches have been mostly lacking. We posit two reasons for this: First, the blur kernel is often only known at test time, requiring any discriminative approach to cope with considerable variability. Second, given this variability it is quite difficult to construct suitable features for discriminative prediction. To address these challenges we first show a conn…
V-Bridge: Bridging Video Generative Priors to Versatile Few-shot Image Restoration
The Hong Kong University of Science and Technology（香港科学与技术大学）；Harbin Institute of Technology, Shenzhen（哈尔滨工业大学（深圳））；The Chinese University of Hong Kong（香港中文大学）
LoRA-IR: Taming Low-Rank Experts for Efficient All-in-One Image Restoration
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）
Hierarchical Information Flow for Generalized Efficient Image Restoration
ETH Zürich（苏黎世联邦理工学院）；University of Pisa（比萨大学）；University of Trento（特伦托大学）；Meta Reality Labs（元宇宙现实实验室）；Peking University（北京大学）；University of California, Merced（加州大学默塞德分校）
GridFormer: Residual Dense Transformer with Grid Structure for Image Restoration in Adverse Weather Conditions
Harbin Institute of Technology, Shenzhen（哈尔滨工业大学（深圳））；Hong Kong University of Science and Technology（香港科技大学）；Rakuten Institute of Technology（乐天理工研究院）；Imperial College London（伦敦帝国学院）；KAIST（韩国科学技术院）；Tencent（腾讯）；Australian National University（澳大利亚国立大学）；Nanjing University（南京大学）
With the recent increase in intelligent CCTVs for visual surveillance, a new image degradation that integrates resolution conversion and synthetic rain models is required. For example, in heavy rain, face images captured by CCTV from a distance have significant deterioration in both visibility and resolution. Unlike traditional image degradation models (IDM), such as rain removal and superresolution, this study addresses a new IDM referred to as a scale-aware heavy rain model and proposes a method for restoring high-resolution face images (HR-FIs) from low-resolution heavy rain face images (LR…
URWKV: Unified RWKV Model with Multi-state Perspective for Low-light Image Restoration
Fujian Key Laboratory of Network Computing and Intelligent Information Processing, College of Computer and Data Science, Fuzhou University（福建网络计算与智能信息处理重点实验室，计算机与数据科学学院，福州大学）；Engineering Research Center of Big Data Intelligence, Ministry of Education（大数据智能工程研究中心，教育部）
Degradation Frequency Curve: An Explicit Frequency-Quantified Representation for All-in-One Image Restoration
College of Electronic Science, National University of Defense Technology（国防科技大学电子科学学院）；College of Electronic Engineering, Beihang University（北航电子工程学院）；VCIP, School of Computer Science, Nankai University（南开大学计算机科学学院）
Burstormer: Burst Image Restoration and Enhancement Transformer
Mohamed bin Zayed University of AI（穆罕默德·本·扎耶德人工智能大学）；Inception Institute of AI（Inception人工智能研究院）；Australian National University（澳大利亚国立大学）；Linköping University（林雪平大学）；University of California, Merced（加州大学默塞德分校）；Yonsei University（延世大学）；Google Research（谷歌研究院）
