Diffusion models have achieved remarkable progress in generative modelling, particularly in enhancing image quality to conform to human preferences. Recently, these models have also been applied to low-level computer vision for photo-realistic image restoration (IR) in tasks such as image denoising, deblurring, dehazing, etc. In this review paper, we introduce key constructions in diffusion models and survey contemporary techniques that make use of diffusion models in solving general IR tasks. Furthermore, we point out the main challenges and limitations of existing diffusion-based IR frameworks and provide potential directions for future work.
LoRA-IR: Taming Low-Rank Experts for Efficient All-in-One Image Restoration
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）
DiffVSR: Revealing an Effective Recipe for Taming Robust Video Super-Resolution Against Complex Degradations
Shanghai Jiao Tong University（上海交通大学）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；University of Science and Technology of China（中国科学技术大学）；Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）
Deep Learning-Driven Ultra-High-Definition Image Restoration: A Survey
School of Mathematical Sciences, Dalian University of Technology（大连理工大学数学科学学院）；Centre for Advances in Reliability and Safety, Hong Kong, China（香港可靠性与安全中心）；Hong Kong Polytechnic University（香港理工大学）；School of Computer Science and Engineering, Nanjing University of Science and Technology（南京理工大学计算机科学与工程学院）
Academy for Engineering & Technology, Fudan University（复旦大学工程与应用技术研究院）；College of Information Technology, Shanghai Ocean University（上海海洋大学信息学院）；University of Calabria（卡拉布里亚大学）；School of Computer Science, Fudan University（复旦大学计算机科学学院）；Napier Edinburgh University（爱丁堡龙比亚大学）
Underwater images play a key role in ocean exploration, but often suffer from severe quality degradation due to light absorption and scattering in water medium. Although major breakthroughs have been made recently in the general area of image enhancement and restoration, the applicability of new methods for improving the quality of underwater images has not specifically been captured. In this paper, we review the image enhancement and restoration methods that tackle typical underwater image impairments, including some extreme degradations and distortions. Firstly, we introduce the key causes o…
Transform image processing methods are methods that work in domains of image transforms, such as Discrete Fourier, Discrete Cosine, Wavelet and alike. They are the basic tool in image compression, in image restoration, in image re-sampling and geometrical transformations and can be traced back to early 1970-ths. The paper presents a review of these methods with emphasis on their comparison and relationships, from the very first steps of transform image compression methods to adaptive and local adaptive transform domain filters for image restoration, to methods of precise image re-sampling and …
The State Key Lab for Novel Software Technology, Nanjing University（新型软件技术国家重点实验室，南京大学）；Australian National University（澳大利亚国立大学）；Imperial College London（伦敦帝国理工学院）；Tencent（腾讯）
Diffusion Models for Image Restoration and Enhancement: A Comprehensive Survey
University of Science and Technology of China（科学技术大学）；National University of Singapore（国立新加坡大学）；Eastern Institute for Advanced Study（东部高级研究机构）；Microsoft Research Asia（微软亚洲研究院）
Centre for Intelligent Sensing, Queen Mary University of London（伦敦玛丽女王大学智能感知中心）
Underwater images are degraded by the selective attenuation of light that distorts colours and reduces contrast. The degradation extent depends on the water type, the distance between an object and the camera, and the depth under the water surface the object is at. Underwater image filtering aims to restore or to enhance the appearance of objects captured in an underwater image. Restoration methods compensate for the actual degradation, whereas enhancement methods improve either the perceived image quality or the performance of computer vision algorithms. The growing interest in underwater ima…
Over the last decade, it has been demonstrated that many systems in science and engineering can be modeled more accurately by fractional-order than integer-order derivatives, and many methods are developed to solve the problem of fractional systems. Due to the extra free parameter order, fractional-order based methods provide additional degree of freedom in optimization performance. Not surprisingly, many fractional-order based methods have been used in image processing field. Herein recent studies are reviewed in ten sub-fields, which include image enhancement, image denoising, image edge det…
Motion blur is one of the most common degradation artifacts in dynamic scene photography. This paper reviews the NTIRE 2020 Challenge on Image and Video Deblurring. In this challenge, we present the evaluation results from 3 competition tracks as well as the proposed solutions. Track 1 aims to develop single-image deblurring methods focusing on restoration quality. On Track 2, the image deblurring methods are executed on a mobile platform to find the balance of the running speed and the restoration accuracy. Track 3 targets developing video deblurring methods that exploit the temporal relation…
Kyushu Institute of Technology（九州工业大学）；Chinese Academy of Science（中国科学院）；Yangzhou University（扬州大学）；Nanjing Normal University（南京师范大学）；Huazhong University of Science and Technology（华中科技大学）
Underwater cameras are widely used to observe the sea floor. They are usually included in autonomous underwater vehicles, unmanned underwater vehicles, and in situ ocean sensor networks. Despite being an important sensor for monitoring underwater scenes, there exist many issues with recent underwater camera sensors. Because of lights transportation characteristics in water and the biological activity at the sea floor, the acquired underwater images often suffer from scatters and large amounts of noise. Over the last five years, many methods have been proposed to overcome traditional underwater…
Nanyang Technological University（南洋理工大学）；SenseBrain；Shanghai AI Laboratory（上海人工智能实验室）；SenseTime Research（商汤科技研究院）
Developing and integrating advanced image sensors with novel algorithms in camera systems are prevalent with the increasing demand for computational photography and imaging on mobile platforms. However, the lack of high-quality data for research and the rare opportunity for in-depth exchange of views from industry and academia constrain the development of mobile intelligent photography and imaging (MIPI). To bridge the gap, we introduce the first MIPI challenge including five tracks focusing on novel image sensors and imaging algorithms. In this paper, we summarize and review the Under-Display…
Remote Sensing Image Dehazing: A Systematic Review of Progress, Challenges, and Prospects
School of Artificial Intelligence and Computer Science（人工智能与计算机科学学院）；Josef Kittler Research Institute on Artificial Intelligence（乔塞夫·基特勒人工智能研究所）；School of Electronic Engineering（电子工程学院）；College of Artificial Intelligence（人工智能学院）；Aerospace Information Research Institute（航天信息研究所）；Department of Computer Science（计算机科学系）
Centre for Quantum Computation & Intelligent Systems（量子计算与智能系统中心）；Faculty of Engineering & Information Technology（工程与信息技术学院）；University of Technology Sydney（悉尼科技大学）
This paper comprehensively reviews the recent development of image deblurring, including non-blind/blind, spatially invariant/variant deblurring techniques. Indeed, these techniques share the same objective of inferring a latent sharp image from one or several corresponding blurry images, while the blind deblurring techniques are also required to derive an accurate blur kernel. Considering the critical role of image restoration in modern imaging systems to provide high-quality images under complex environments such as motion, undesirable lighting conditions, and imperfect system components, im…
A review of Bayesian restoration of digital images based on Monte Carlo techniques is presented. The topics covered include Likelihood, Prior and Posterior distributions, Poisson, Binay symmetric channel, and Gaussian channel models of Likelihood distribution,Ising and Potts spin models of Prior distribution, restoration of an image through Posterior maximization, statistical estimation of a true image from Posterior ensembles, Markov Chain Monte Carlo methods and cluster algorithms.
The appearance of objects in underwater images is degraded by the selective attenuation of light, which reduces contrast and causes a colour cast. This degradation depends on the water environment, and increases with depth and with the distance of the object from the camera. Despite an increasing volume of works in underwater image enhancement and restoration, the lack of a commonly accepted evaluation measure is hindering the progress as it is difficult to compare methods. In this paper, we review commonly used colour accuracy measures, such as colour reproduction error and CIEDE2000, and no-…
This paper reviews the first challenge on high-dynamic range (HDR) imaging that was part of the New Trends in Image Restoration and Enhancement (NTIRE) workshop, held in conjunction with CVPR 2021. This manuscript focuses on the newly introduced dataset, the proposed methods and their results. The challenge aims at estimating a HDR image from one or multiple respective low-dynamic range (LDR) observations, which might suffer from under- or over-exposed regions and different sources of noise. The challenge is composed by two tracks: In Track 1 only a single LDR image is provided as input, where…
Aerospace Information Research Institute, Chinese Academy of Sciences（中国科学院空天信息创新研究院）；University of Chinese Academy of Sciences（中国科学院大学）；International Research Center of Big Data for Sustainable Development Goals（可持续发展大数据国际研究中心）
Owing to the rapid development of sensor technology, hyperspectral (HS) remote sensing (RS) imaging has provided a significant amount of spatial and spectral information for the observation and analysis of the Earth's surface at a distance of data acquisition devices, such as aircraft, spacecraft, and satellite. The recent advancement and even revolution of the HS RS technique offer opportunities to realize the full potential of various applications, while confronting new challenges for efficiently processing and analyzing the enormous HS acquisition data. Due to the maintenance of the 3-D HS …
AuthFace: Towards Authentic Blind Face Restoration with Face-oriented Generative Diffusion Prior
Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；vivo Mobile Communication Co., Ltd（vivo移动通信有限公司）；Nanyang Technological University（南洋理工大学）
Clear Roads, Clear Vision: Advancements in Multi-Weather Restoration for Smart Transportation
Finolex Academy of Management and Technology（Finolex管理与技术学院）；Drone Lab, Centre for Artificial Intelligence and Robotics, IIT Mandi（印度理工学院曼迪分校人工智能与机器人中心无人机实验室）；Mehta Family School of Data Science and Artificial Intelligence, IIT Guwahati（印度理工学院古瓦哈提分校梅塔家族数据科学与人工智能学院）；SPACE42；Centre of Excellence: Artificial Intelligence, School of Computer Science, UPES Dehradun（UPES德拉敦分校计算机科学学院人工智能卓越中心）
Harbin Institute of Technology（哈尔滨工业大学）；Hong Kong Polytechnic University（香港理工大学）
Generative adversarial networks (GANs) have drawn enormous attention due to the simple yet effective training mechanism and superior image generation quality. With the ability to generate photo-realistic high-resolution (e.g., $1024\times1024$) images, recent GAN models have greatly narrowed the gaps between the generated images and the real ones. Therefore, many recent works show emerging interest to take advantage of pre-trained GAN models by exploiting the well-disentangled latent space and the learned GAN priors. In this paper, we briefly review recent progress on leveraging pre-trained la…
A Comprehensive Survey of Mamba Architectures for Medical Image Analysis: Classification, Segmentation, Restoration and Beyond
Indian Institute of Technology Indore, India；R.M.D. Engineering College, Kavaraipettai, India；Jio Platforms Limited, India；Birla Institute of Technology \& Science Pilani, India
MESA: A Training-Free Multi-Exemplar Deep Framework for Restoring Ancient Inscription Textures
Department of Computer Science and Engineering（计算机科学与工程系）；University of Ioannina（伊奥安纳大学）；Inria Paris-Rocquencourt（巴黎-罗克琴堡研究所）；Rajiv Gandhi University（拉朱·甘地大学）；Tsinghua University（清华大学）；Palmer Research Laboratories（帕勒尔研究实验室）
Deakin University（迪肯大学）；Shenzhen University（深圳大学）；Dibrugarh University（迪布鲁格尔大学）；Swinburne University of Technology（斯威本科技大学）；University of Oulu（奥卢大学）；Google Research（谷歌研究院）；University of Waterloo（滑铁卢大学）；Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）
Uncertainty quantification (UQ) plays a pivotal role in reduction of uncertainties during both optimization and decision making processes. It can be applied to solve a variety of real-world applications in science and engineering. Bayesian approximation and ensemble learning techniques are two most widely-used UQ methods in the literature. In this regard, researchers have proposed different UQ methods and examined their performance in a variety of applications such as computer vision (e.g., self-driving cars and object detection), image processing (e.g., image restoration), medical image analy…
Medical imaging is crucial in modern clinics to guide the diagnosis and treatment of diseases. Medical image reconstruction is one of the most fundamental and important components of medical imaging, whose major objective is to acquire high-quality medical images for clinical usage at minimal cost and risk to the patients. Mathematical models in medical image reconstruction or, more generally, image restoration in computer vision, have been playing a prominent role. Earlier mathematical models are mostly designed by human knowledge or hypothesis on the image to be reconstructed, and we shall c…
