The availability of large-scale datasets has helped unleash the true potential of deep convolutional neural networks (CNNs). However, for the single-image denoising problem, capturing a real dataset is an unacceptably expensive and cumbersome procedure. Consequently, image denoising algorithms are mostly developed and evaluated on synthetic data that is usually generated with a widespread assumption of additive white Gaussian noise (AWGN). While the CNNs achieve impressive results on these synthetic datasets, they do not perform well when applied on real camera images, as reported in recent benchmark datasets. This is mainly because the AWGN is not adequate for modeling the real camera noise which is signal-dependent and heavily transformed by the camera imaging pipeline. In this paper, we present a framework that models camera imaging pipeline in forward and reverse directions. It allows us to produce any number of realistic image pairs for denoising both in RAW and sRGB spaces. By training a new image denoising network on realistic synthetic data, we achieve the state-of-the-art performance on real camera benchmark datasets. The parameters in our model are ~5 times lesser than the previous best method for RAW denoising. Furthermore, we demonstrate that the proposed framework generalizes beyond image denoising problem e.g., for color matching in stereoscopic cinema. The source code and pre-trained models are available at https://github.com/swz30/CycleISP.
Tsinghua University（清华大学）；Shenzhen Institutes of Advanced Technology (SIAT), Chinese Academy of Science（中国科学院深圳先进技术研究院）；Tsinghua Shenzhen International Graduate School（清华大学深圳国际研究生院）；University College London（伦敦大学学院）
Most state-of-the-art semantic segmentation approaches only achieve high accuracy in good conditions. In practically-common but less-discussed adverse environmental conditions, their performance can decrease enormously. Existing studies usually cast the handling of segmentation in adverse conditions as a separate post-processing step after signal restoration, making the segmentation performance largely depend on the quality of restoration. In this paper, we propose a novel deep-learning framework to tackle semantic segmentation and image restoration in adverse environmental conditions in a hol…
GenSmoke-GS: A Multi-Stage Method for Novel View Synthesis from Smoke-Degraded Images Using a Generative Model
School of Computer Science and Technology, Hangzhou Dianzi University（杭州电子科技大学计算机科学与技术学院）；School of Computer Science and Technology, Harbin Institute of Technology, Shenzhen（哈尔滨工业大学（深圳）计算机科学与技术学院）
Blind image deblurring remains a challenging problem for modern artificial neural networks. Unlike other image restoration problems, deblurring networks fail behind the performance of existing deblurring algorithms in case of uniform and 3D blur models. This follows from the diverse and profound effect that the unknown blur-kernel has on the deblurring operator. We propose a new architecture which breaks the deblurring network into an analysis network which estimates the blur, and a synthesis network that uses this kernel to deblur the image. Unlike existing deblurring networks, this design al…
From Pixels to Pathology: Restoration Diffusion for Diagnostic-Consistent Virtual IHC
organization= Institute of Pathology, Technical University of Munich, TUM School of Medicine；organization= Computer Aided Medical Procedures (CAMP), TU Munich , city= Munich , country= Germany；organization= Munich Center for Machine Learning (MCML) , city= Munich , country= Germany；organization= TUM School of Computation, Information；organization= Department of Medical Informatics, School of Medicine, Mashhad University of Medical Sciences , city= Mashhad , country= Iran；organization= Munich Data Science Institute (MDSI) , city= Munich , country= Germany
HIR-ALIGN: Enhancing Hyperspectral Image Restoration via Diffusion-Based Data Generation
School of Mathematics and Statistics, Xi’an Jiaotong University（西安交通大学数学与统计学学院）；School of Computer Science and Technology, Xi’an Jiaotong University（西安交通大学计算机科学与技术学院）；School of Mathematics and Statistics and the Ministry of Education Key Laboratory for Intelligent Networks and Network Security, Xi’an Jiaotong University（西安交通大学数学与统计学学院和教育部智能网络与网络安全重点实验室）；Pazhou Laboratory (Huangpu), Guangzhou（广州黄埔 Pazhou 实验室）
All-in-One Video Restoration under Smoothly Evolving Unknown Weather Degradations
Department of Computer Science and Technology, Harbin Institute of Technology（计算机科学与技术系，哈尔滨工业大学）；Peng Cheng Laboratory, Shenzhen, China（鹏城实验室，深圳，中国）；School of Computer Science, Peking University（计算机学院，北京大学）
All-in-One Medical Image Restoration with Latent Diffusion-Enhanced Vector-Quantized Codebook Prior
School of Biological Science and Medical Engineering（生物科学与医学工程学院）；State Key Laboratory of Software Development Environment（软件开发环境国家重点实验室）；Key Laboratory of Biomechanics and Mechanobiology of Ministry of Education（教育部生物力学与机械生物学重点实验室）；Beijing Advanced Innovation Center for Biomedical Engineering（北京生物医学工程先进创新中心）；Beihang University（北京航空航天大学）；Department of Biomedical Engineering（生物医学工程系）；Tsinghua University（清华大学）；ByteDance Inc.（字节跳动公司）
Robust Low-light Scene Restoration via Illumination Transition
The Hong Kong University of Science and Technology（香港科学与技术大学）；The Hong Kong Polytechnic University（香港理工大学）；Nanjing University of Posts and Telecommunications（南京邮电大学）；University of Surrey（萨里大学）
TAT: Task-Adaptive Transformer for All-in-One Medical Image Restoration
School of Biological Science and Medical Engineering（生物科学与医学工程学院）；State Key Laboratory of Software Development Environment（软件开发环境国家重点实验室）；Key Laboratory of Biomechanics and Mechanobiology of Ministry of Education（教育部生物力学与机力学重点实验室）；Beijing Advanced Innovation Center for Biomedical Engineering（北京生物医学工程先进创新中心）；Beihang University（北京航空航天大学）；Sinovision Technologies (Beijing) Co., Ltd.（北京视界科技有限公司）；ByteDance Inc.（字节跳动公司）
Indian Institute of Technology Gandhinagar（甘地纳加尔印度理工学院）
Blind motion deblurring involves reconstructing a sharp image from an observation that is blurry. It is a problem that is ill-posed and lies in the categories of image restoration problems. The training data-based methods for image deblurring mostly involve training models that take a lot of time. These models are data-hungry i.e., they require a lot of training data to generate satisfactory results. Recently, there are various image feature learning methods developed which relieve us of the need for training data and perform image restoration and image synthesis, e.g., DIP, InGAN, and SinGAN.…
Analysis and Synthesis Denoisers for Forward-Backward Plug-and-Play Algorithms
Laboratoire Interdisciplinaire des Sciences du Numériques, Inria, Université Paris-Saclay, CNRS, Gif-sur-Yvette, France（跨学科数字科学实验室，法国国家信息与自动化研究所，巴黎-萨克雷大学，法国国家科学研究中心，法国吉夫-sur-伊夫特）；Inria, Université Paris-Saclay, CEA, Palaiseau, France（法国国家信息与自动化研究所，巴黎-萨克雷大学，法国原子能委员会，法国帕莱索）；School of Mathematics and Computer Sciences and School of Engineering and Physical Sciences, Heriot-Watt University, Edinburgh, UK（数学与计算机科学学院和工程与物理科学学院，赫里奥特-瓦特大学，英国爱丁堡）；Maxwell Institute for Mathematical Sciences, Edinburgh, UK（数学科学麦克斯韦研究所，英国爱丁堡）
Applied Research Center (ARC), Tencent PCG（腾讯PCG应用研究中心）；Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）；University of Chinese Academy of Sciences（中国科学院大学）；Shanghai AI Laboratory（上海人工智能实验室）
Though many attempts have been made in blind super-resolution to restore low-resolution images with unknown and complex degradations, they are still far from addressing general real-world degraded images. In this work, we extend the powerful ESRGAN to a practical restoration application (namely, Real-ESRGAN), which is trained with pure synthetic data. Specifically, a high-order degradation modeling process is introduced to better simulate complex real-world degradations. We also consider the common ringing and overshoot artifacts in the synthesis process. In addition, we employ a U-Net discrim…
Wuhan University of Science and Technology（武汉科技大学）；Institute for Infocomm Research（信息通信研究院）
There are shadow and highlight regions in a low dynamic range (LDR) image which is captured from a high dynamic range (HDR) scene. It is an ill-posed problem to restore the saturated regions of the LDR image. In this paper, the saturated regions of the LDR image are restored by fusing model-based and data-driven approaches. With such a neural augmentation, two synthetic LDR images are first generated from the underlying LDR image via the model-based approach. One is brighter than the input image to restore the shadow regions and the other is darker than the input image to restore the high-ligh…
Bridge Damage Detection from Low-Light UAV Imagery via Degradation-Aware Mixture-of-Experts Enhancement
School of Computer Science and Engineering, University of Electronic Science and Technology of China（电子科技大学计算机科学与工程学院）；Sustainability X-Lab, The University of Hong Kong（香港大学可持续发展X实验室）；School of Architecture, Building, and Civil Engineering, Loughborough University（拉夫堡大学建筑、建造与土木工程学院）；Department of Engineering Science, University of Oxford（牛津大学工程科学系）；Department of Civil and Environmental Engineering, The Hong Kong University of Science and Technology（香港科技大学土木与环境工程系）
Recent development of Under-Display Camera (UDC) systems provides a true bezel-less and notch-free viewing experience on smartphones (and TV, laptops, tablets), while allowing images to be captured from the selfie camera embedded underneath. In a typical UDC system, the microstructure of the semi-transparent organic light-emitting diode (OLED) pixel array attenuates and diffracts the incident light on the camera, resulting in significant image quality degradation. Oftentimes, noise, flare, haze, and blur can be observed in UDC images. In this work, we aim to analyze and tackle the aforemention…
University of Illinois, Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
Recent works on adaptive sparse and on low-rank signal modeling have demonstrated their usefulness in various image / video processing applications. Patch-based methods exploit local patch sparsity, whereas other works apply low-rankness of grouped patches to exploit image non-local structures. However, using either approach alone usually limits performance in image reconstruction or recovery applications. In this work, we propose a simultaneous sparsity and low-rank model, dubbed STROLLR, to better represent natural images. In order to fully utilize both the local and non-local image properti…
tabular c Indian Institute of Technology Gandhinagar, India. Ashoka University, Sonipat, India. Larsen \& Toubro Technology Services Limited. Indian Institute of Technology (BHU), Varanasi, India. tabular
University of Science and Technology of China（中国科学技术大学）；Harbin Institute of Technology（哈尔滨工业大学）；Huawei Noah’s Ark Lab（华为诺亚方舟实验室）；University of Macau（澳门大学）；Shanghai AI Laboratory（上海人工智能实验室）；Huawei Cloud BU（华为云业务部）
Low-light video enhancement (LLVE) is an important yet challenging task with many applications such as photographing and autonomous driving. Unlike single image low-light enhancement, most LLVE methods utilize temporal information from adjacent frames to restore the color and remove the noise of the target frame. However, these algorithms, based on the framework of multi-frame alignment and enhancement, may produce multi-frame fusion artifacts when encountering extreme low light or fast motion. In this paper, inspired by the low latency and high dynamic range of events, we use synthetic events…
DDR: Exploiting Deep Degradation Response as Flexible Image Descriptor
School of Computer Science and Technology, Tongji University（同济大学计算机科学与技术学院）；University of California, Santa Cruz（加州大学圣克鲁兹分校）；Pengcheng Laboratory（鹏城实验室）；City University of Hong Kong（香港城市大学）
Continuous Representation Methods, Theories, and Applications: An Overview and Perspectives
School of Mathematics and Statistics, Xi’an Jiaotong University, Xi’an 710000, China（西安交通大学数学与统计学院）；School of Mathematical Sciences, University of Electronic Science and Technology of China, Chengdu 610000, China（电子科技大学数学科学学院）
Northwestern Polytechnical University（西北工业大学）；Computer Vision Center（计算机视觉中心）；Universitat Autònoma de Barcelona（巴塞罗那自治大学）
Data-driven paradigms using machine learning are becoming ubiquitous in image processing and communications. In particular, image-to-image (I2I) translation is a generic and widely used approach to image processing problems, such as image synthesis, style transfer, and image restoration. At the same time, neural image compression has emerged as a data-driven alternative to traditional coding approaches in visual communications. In this paper, we study the combination of these two paradigms into a joint I2I compression and translation framework, focusing on multi-domain image synthesis. We firs…
University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；Food and Drug Administration（美国食品药品监督管理局）
Modern generative models, such as generative adversarial networks (GANs), hold tremendous promise for several areas of medical imaging, such as unconditional medical image synthesis, image restoration, reconstruction and translation, and optimization of imaging systems. However, procedures for establishing stochastic image models (SIMs) using GANs remain generic and do not address specific issues relevant to medical imaging. In this work, canonical SIMs that simulate realistic vessels in angiography images are employed to evaluate procedures for establishing SIMs using GANs. The GAN-based SIM …
University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；Center for Devices and Radiological Health, Food and Drug Administration（美国食品药品监督管理局设备与放射健康中心）
In recent years, generative adversarial networks (GANs) have gained tremendous popularity for potential applications in medical imaging, such as medical image synthesis, restoration, reconstruction, translation, as well as objective image quality assessment. Despite the impressive progress in generating high-resolution, perceptually realistic images, it is not clear if modern GANs reliably learn the statistics that are meaningful to a downstream medical imaging application. In this work, the ability of a state-of-the-art GAN to learn the statistics of canonical stochastic image models (SIMs) t…
