Learning Enriched Features for Real Image Restoration and Enhancement
Inception Institute of Artificial Intelligence（inception人工智能研究院）；Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）；University of California, Merced（加州大学默塞德分校）；Google Research（谷歌研究院）
Syed Waqas Zamir, Aditya Arora, Salman Khan, Munawar Hayat, Fahad Shahbaz Khan, Ming-Hsuan Yang, Ling Shao
With the goal of recovering high-quality image content from its degraded version, image restoration enjoys numerous applications, such as in surveillance, computational photography, medical imaging, and remote sensing. Recently, convolutional neural networks (CNNs) have achieved dramatic improvements over conventional approaches for image restoration task. Existing CNN-based methods typically operate either on full-resolution or on progressively low-resolution representations. In the former case, spatially precise but contextually less robust results are achieved, while in the latter case, semantically reliable but spatially less accurate outputs are generated. In this paper, we present a novel architecture with the collective goals of maintaining spatially-precise high-resolution representations through the entire network and receiving strong contextual information from the low-resolution representations. The core of our approach is a multi-scale residual block containing several key elements: (a) parallel multi-resolution convolution streams for extracting multi-scale features, (b) information exchange across the multi-resolution streams, (c) spatial and channel attention mechanisms for capturing contextual information, and (d) attention based multi-scale feature aggregation. In a nutshell, our approach learns an enriched set of features that combines contextual information from multiple scales, while simultaneously preserving the high-resolution spatial details. Extensive experiments on five real image benchmark datasets demonstrate that our method, named as MIRNet, achieves state-of-the-art results for a variety of image processing tasks, including image denoising, super-resolution, and image enhancement. The source code and pre-trained models are available at https://github.com/swz30/MIRNet.
Inception Institute of Artificial Intelligence（起源人工智能研究院）；Mohammed Bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）；University of California at Merced（加州大学默塞德分校）；Google（谷歌）；Terminus Group（特斯联集团）
Given a degraded input image, image restoration aims to recover the missing high-quality image content. Numerous applications demand effective image restoration, e.g., computational photography, surveillance, autonomous vehicles, and remote sensing. Significant advances in image restoration have been made in recent years, dominated by convolutional neural networks (CNNs). The widely-used CNN-based methods typically operate either on full-resolution or on progressively low-resolution representations. In the former case, spatial details are preserved but the contextual information cannot be prec…
University of Notre Dame（圣母大学）；University of Ljubljana（卢布尔雅那大学）
Advances in image restoration and enhancement techniques have led to discussion about how such algorithmscan be applied as a pre-processing step to improve automatic visual recognition. In principle, techniques like deblurring and super-resolution should yield improvements by de-emphasizing noise and increasing signal in an input image. But the historically divergent goals of the computational photography and visual recognition communities have created a significant need for more work in this direction. To facilitate new research, we introduce a new benchmark dataset called UG^2, which contain…
Neural Discrimination-Prompted Transformers for Efficient UHD Image Restoration and Enhancement
Shenzhen Campus of Sun Yat-Sen University（中山大学深圳校区）；The Hong Kong Polytechnic University（香港理工大学）；Nanjing University of Science and Technology（南京理工大学）；Dalian University of Technology（大连理工大学）；University of California, San Francisco（加州大学旧金山分校）
Nanjing University of Aeronautics and Astronautics（南京航空航天大学）；Lingnan University（岭南大学）；Hong Kong Metropolitan University（香港都会大学）
Adverse weather conditions such as haze, rain, and snow often impair the quality of captured images, causing detection networks trained on normal images to generalize poorly in these scenarios. In this paper, we raise an intriguing question - if the combination of image restoration and object detection, can boost the performance of cutting-edge detectors in adverse weather conditions. To answer it, we propose an effective yet unified detection paradigm that bridges these two subtasks together via dynamic enhancement learning to discern objects in adverse weather conditions, called TogetherNet.…
A Generative Data Framework with Authentic Supervision for Underwater Image Restoration and Enhancement
Institute of Artificial Intelligence (TeleAI), China Telecom（人工智能研究院（TeleAI），中国电信）；Department of Automation, Shanghai Jiao Tong University（自动化系，上海交通大学）；School of Artificial Intelligence, Optics and ElectroNics (iOPEN), Northwestern Polytechnical University（人工智能、光学与电子学学院（iOPEN），西北工业大学）；Xi’an Institute of Optics and Precision Mechanics, Chinese Academy of Sciences（西安光学精密机械研究所，中国科学院）
Hanyang University（汉阳大学）；LG AI Research（LG人工智能研究院）
Recent transformer-based super-resolution (SR) methods have achieved promising results against conventional CNN-based methods. However, these approaches suffer from essential shortsightedness created by only utilizing the standard self-attention-based reasoning. In this paper, we introduce an effective hybrid SR network to aggregate enriched features, including local features from CNNs and long-range multi-scale dependencies captured by transformers. Specifically, our network comprises transformer and convolutional branches, which synergetically complement each representation during the restor…
Diffusion Models for Image Restoration and Enhancement: A Comprehensive Survey
University of Science and Technology of China（科学技术大学）；National University of Singapore（国立新加坡大学）；Eastern Institute for Advanced Study（东部高级研究机构）；Microsoft Research Asia（微软亚洲研究院）
Academy for Engineering & Technology, Fudan University（复旦大学工程与应用技术研究院）；College of Information Technology, Shanghai Ocean University（上海海洋大学信息学院）；University of Calabria（卡拉布里亚大学）；School of Computer Science, Fudan University（复旦大学计算机科学学院）；Napier Edinburgh University（爱丁堡龙比亚大学）
Underwater images play a key role in ocean exploration, but often suffer from severe quality degradation due to light absorption and scattering in water medium. Although major breakthroughs have been made recently in the general area of image enhancement and restoration, the applicability of new methods for improving the quality of underwater images has not specifically been captured. In this paper, we review the image enhancement and restoration methods that tackle typical underwater image impairments, including some extreme degradations and distortions. Firstly, we introduce the key causes o…
CRNet: A Detail-Preserving Network for Unified Image Restoration and Enhancement Task
Northwestern Polytechnical University（西北工业大学）；Xi’an University of Architecture and Technology（西安建筑科技大学）；Xi’an Institute of Optics and Precision Mechanics of CAS（中国科学院西安光学精密机械研究所）
Scaling Up to Excellence: Practicing Model Scaling for Photo-Realistic Image Restoration In the Wild
Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）；Shanghai AI Laboratory（上海人工智能实验室）；The Hong Kong Polytechnic University（香港理工大学）；ARC Lab, Tencent PCG（腾讯平台与内容事业群ARC实验室）；The Chinese University of Hong Kong（香港中文大学）
Bracketing Image Restoration and Enhancement with High-Low Frequency Decomposition
Xi’an University of Architecture and Technology（西安建筑科技大学）；Northwestern Polytechnical University（西北工业大学）；University of Macau（澳门大学）；Xi’an Institute of Optics and Precision Mechanics of CAS（中国科学院西安光学精密机械研究所）
Gated Multi-Resolution Transfer Network for Burst Restoration and Enhancement
Indian Institute of Technology Ropar（印度理工学院鲁尔基分校）；Mohamed bin Zayed University of AI（穆罕默德·本·扎耶德人工智能大学）；Inception Institute of AI（起源人工智能研究院）；Australian National University（澳大利亚国立大学）；Linköping University（林雪平大学）
National Tsing Hua University（国立清华大学）；Chalmers University of Technology（查尔姆斯理工大学）；Universiti Malaya（马来亚大学）；Microsoft AI R & D Center（微软人工智能研发中心）
Deep learning-based methods have made impressive progress in enhancing extremely low-light images - the image quality of the reconstructed images has generally improved. However, we found out that most of these methods could not sufficiently recover the image details, for instance, the texts in the scene. In this paper, a novel image enhancement framework is proposed to precisely restore the scene texts, as well as the overall quality of the image simultaneously under extremely low-light images conditions. Mainly, we employed a self-regularised attention map, an edge map, and a novel text dete…
Boosting Image Restoration via Priors from Pre-trained Models
Zhejiang Lab（之江实验室）；CUHK（香港中文大学）；RealityEdge（睿迪智芯）；University of Macau（澳门大学）；Institute of Collaborative Innovation（协同创新研究院）；Texas A&M University（德克萨斯农工大学）；National University of Singapore（新加坡国立大学）；Zhejiang University（浙江大学）
Mohamed bin Zayed University of AI（穆罕默德·本·扎耶德人工智能大学）；Inception Institute of AI（英赛普申人工智能研究院）；Australian National University（澳大利亚国立大学）；Linköping University（林雪平大学）；University of California, Merced（加州大学默塞德分校）；Yonsei University（延世大学）；Google Research（谷歌研究院）
Modern handheld devices can acquire burst image sequence in a quick succession. However, the individual acquired frames suffer from multiple degradations and are misaligned due to camera shake and object motions. The goal of Burst Image Restoration is to effectively combine complimentary cues across multiple burst frames to generate high-quality outputs. Towards this goal, we develop a novel approach by solely focusing on the effective information exchange between burst frames, such that the degradations get filtered out while the actual scene details are preserved and enhanced. Our central id…
Southern University of Science and Technology（南方科技大学）；A*STAR（新加坡科技研究局）；New York University（纽约大学）；Chinese Academy of Sciences（中国科学院）；Shenzhen People’s Hospital（深圳市人民医院）
Fundus photography is a routine examination in clinics to diagnose and monitor ocular diseases. However, for cataract patients, the fundus image always suffers quality degradation caused by the clouding lens. The degradation prevents reliable diagnosis by ophthalmologists or computer-aided systems. To improve the certainty in clinical diagnosis, restoration algorithms have been proposed to enhance the quality of fundus images. Unfortunately, challenges remain in the deployment of these algorithms, such as collecting sufficient training data and preserving retinal structures. In this paper, to …
Burstormer: Burst Image Restoration and Enhancement Transformer
Mohamed bin Zayed University of AI（穆罕默德·本·扎耶德人工智能大学）；Inception Institute of AI（Inception人工智能研究院）；Australian National University（澳大利亚国立大学）；Linköping University（林雪平大学）；University of California, Merced（加州大学默塞德分校）；Yonsei University（延世大学）；Google Research（谷歌研究院）
Deep Learning-Driven Ultra-High-Definition Image Restoration: A Survey
School of Mathematical Sciences, Dalian University of Technology（大连理工大学数学科学学院）；Centre for Advances in Reliability and Safety, Hong Kong, China（香港可靠性与安全中心）；Hong Kong Polytechnic University（香港理工大学）；School of Computer Science and Engineering, Nanjing University of Science and Technology（南京理工大学计算机科学与工程学院）
Unified Restoration-Perception Learning: Maritime Infrared-Visible Image Fusion and Segmentation
Laboratory of Underwater Acoustic Communication and Marine Information Technology, Ministry of Education, Xiamen University（厦门大学水声通信与海洋信息技术教育部重点实验室）；PAMI Research Group, Department of Computer and Information Science, University of Macau（澳门大学计算机与信息科学系PAMI研究组）；School of Cyber Science and Technology, Shenzhen Campus of Sun Yat-sen University（中山大学深圳校区网络空间安全学院）
Binarized High-Efficiency RAW Video Restoration and Beyond
School of Computer Science and Technology, Beijing Institute of Technology（北京理工大学计算机科学与技术学院）；Chinese Aeronautical Establishment（中国航空研究院）；Westlake University（西湖大学）；Shanghai Jiao Tong University（上海交通大学）
Nanyang Technological University（南洋理工大学）；University of Science and Technology of China（中国科学技术大学）；Nankai University（南开大学）
Image and video restoration has achieved a remarkable leap with the advent of deep learning. The success of deep learning paradigm lies in three key components: data, model, and loss. Currently, many efforts have been devoted to the first two while seldom study focuses on loss function. With the question ``are the de facto optimization functions e.g., $L_1$, $L_2$, and perceptual losses optimal?'', we explore the potential of loss and raise our belief ``learned loss function empowers the learning capability of neural networks for image and video restoration''. Concretely, we stand on the shoul…
Breaking Degradation Coupling: A Structural Entropy Guided Decoupled Framework and Benchmark for Infrared Enhancement
Faculty of Information Engineering and Automation, Kunming University of Science and Technology（昆明理工大学信息工程与自动化学院）；Department of Biomedical Engineering, Hefei University of Technology（合肥工业大学生物医学工程系）；School of Mathematics and Statistics, Yunnan University（云南大学数学与统计学院）
Xi’an University of Science and Technology（西安科技大学）
A novel color image enhancement method is proposed based on Retinex to enhance color images under non-uniform illumination or poor visibility conditions. Different from the conventional Retinex algorithms, the Weighted Guided Image Filter is used as a surround function instead of the Gaussian filter to estimate the background illumination, which can overcome the drawbacks of local blur and halo artifact that may appear by Gaussian filter. To avoid color distortion, the image is converted to the HSI color model, and only the intensity channel is enhanced. Then a linear color restoration algorit…
All existing image enhancement methods, such as HDR tone mapping, cannot recover A/D quantization losses due to insufficient or excessive lighting, (underflow and overflow problems). The loss of image details due to A/D quantization is complete and it cannot be recovered by traditional image processing methods, but the modern data-driven machine learning approach offers a much needed cure to the problem. In this work we propose a novel approach to restore and enhance images acquired in low and uneven lighting. First, the ill illumination is algorithmically compensated by emulating the effects …
Centre for Intelligent Sensing, Queen Mary University of London（伦敦玛丽女王大学智能感知中心）
Underwater images are degraded by the selective attenuation of light that distorts colours and reduces contrast. The degradation extent depends on the water type, the distance between an object and the camera, and the depth under the water surface the object is at. Underwater image filtering aims to restore or to enhance the appearance of objects captured in an underwater image. Restoration methods compensate for the actual degradation, whereas enhancement methods improve either the perceived image quality or the performance of computer vision algorithms. The growing interest in underwater ima…
