A mainstream type of the state of the arts (SOTAs) based on convolutional neural network (CNN) for real image denoising contains two sub-problems, i.e., noise estimation and non-blind denoising. This paper considers real noise approximated by heteroscedastic Gaussian/Poisson Gaussian distributions with in-camera signal processing pipelines. The related works always exploit the estimated noise prior via channel-wise concatenation followed by a convolutional layer with spatially sharing kernels. Due to the variable modes of noise strength and frequency details of all feature positions, this design cannot adaptively tune the corresponding denoising patterns. To address this problem, we propose a novel conditional filter in which the optimal kernels for different feature positions can be adaptively inferred by local features from the image and the noise map. Also, we bring the thought that alternatively performs noise estimation and non-blind denoising into CNN structure, which continuously updates noise prior to guide the iterative feature denoising. In addition, according to the property of heteroscedastic Gaussian distribution, a novel affine transform block is designed to predict the stationary noise component and the signal-dependent noise component. Compared with SOTAs, extensive experiments are conducted on five synthetic datasets and three real datasets, which shows the improvement of the proposed CFNet.
For flexible non-blind image denoising, existing deep networks usually take both noisy image and noise level map as the input to handle various noise levels with a single model. However, in this kind of solution, the noise variance (i.e., noise level) is only deployed to modulate the first layer of convolution feature with channel-wise shifting, which is limited in balancing noise removal and detail preservation. In this paper, we present a novel flexible image enoising network (CFMNet) by equipping an U-Net backbone with multi-layer conditional feature modulation (CFM) modules. In comparison …
University of Maryland, College Park（马里兰大学帕克分校）；Mitsubishi Electric Research Lab.（三菱电机研究实验室）
We propose a novel deep network architecture for image\\ denoising based on a Gaussian Conditional Random Field (GCRF) model. In contrast to the existing discriminative denoising methods that train a separate model for each noise level, the proposed deep network explicitly models the input noise variance and hence is capable of handling a range of noise levels. Our deep network, which we refer to as deep GCRF network, consists of two sub-networks: (i) a parameter generation network that generates the pairwise potential parameters based on the noisy input image, and (ii) an inference network wh…
Huawei Noah’s Ark（华为诺亚方舟实验室）；Imperial College London（帝国理工学院）；University College London（伦敦大学学院）
Deep Convolutional Neural Networks (CNNs) have been successfully used in many low-level vision problems like image denoising. Although the conditional image generation techniques have led to large improvements in this task, there has been little effort in providing conditional generative adversarial networks (cGAN)[42] with an explicit way of understanding the image noise for object-independent denoising reliable for real-world applications. The task of leveraging structures in the target space is unstable due to the complexity of patterns in natural scenes, so the presence of unnatural artifa…
Building Bridges across Spatial and Temporal Resolutions: Reference-Based Super-Resolution via Change Priors and Conditional Diffusion Model
Tsinghua University（清华大学）；Tsinghua University - Xi’an Institute of Surveying and Mapping Joint Research Center（清华大学-西安测绘联合研究中心）；The University of Hong Kong（香港大学）；National Supercomputing Center in Shenzhen（深圳国家超级计算中心）；Sun Yat-Sen University（中山大学）
Convolutional neural networks (CNNs) have shown outstanding performance on image denoising with the help of large-scale datasets. Earlier methods naively trained a single CNN with many pairs of clean-noisy images. However, the conditional distribution of the clean image given a noisy one is too complicated and diverse, so that a single CNN cannot well learn such distributions. Therefore, there have also been some methods that exploit additional noise level parameters or train a separate CNN for a specific noise level parameter. These methods separate the original problem into easier sub-proble…
School of Computer and Communication Sciences, EPFL（洛桑联邦理工学院计算机与通信科学学院）
Super-resolution and denoising are ill-posed yet fundamental image restoration tasks. In blind settings, the degradation kernel or the noise level are unknown. This makes restoration even more challenging, notably for learning-based methods, as they tend to overfit to the degradation seen during training. We present an analysis, in the frequency domain, of degradation-kernel overfitting in super-resolution and introduce a conditional learning perspective that extends to both super-resolution and denoising. Building on our formulation, we propose a stochastic frequency masking of images used in…
PPFM: Image denoising in photon-counting CT using single-step posterior sampling Poisson flow generative models
KTH Royal Institute of Technology（皇家理工学院）；Karolinska University Hospital（卡罗林斯卡大学医院）；University of Wisconsin School of Medicine and Public Health（威斯康星大学麦迪逊分校医学与公共卫生学院）；GE HealthCare（GE医疗）；Rensselaer Polytechnic Institute（伦斯勒理工学院）
We present SR3, an approach to image Super-Resolution via Repeated Refinement. SR3 adapts denoising diffusion probabilistic models to conditional image generation and performs super-resolution through a stochastic denoising process. Inference starts with pure Gaussian noise and iteratively refines the noisy output using a U-Net model trained on denoising at various noise levels. SR3 exhibits strong performance on super-resolution tasks at different magnification factors, on faces and natural images. We conduct human evaluation on a standard 8X face super-resolution task on CelebA-HQ, comparing…
This paper proposes a general framework for internal patch-based image restoration based on Conditional Random Fields (CRF). Unlike related models based on Markov Random Fields (MRF), our approach explicitly formulates the posterior distribution for the entire image. The potential functions are taken as proportional to the product of a likelihood and prior for each patch. By assuming identical parameters for similar patches, our approach can be classified as a model-based non-local method. For the prior term in the potential function of the CRF model, multivariate Gaussians and multivariate sc…
The potential of using remote sensing imagery for environmental modelling and for providing real time support to humanitarian operations such as hurricane relief efforts is well established. These applications are substantially affected by missing data due to non-structural noise such as clouds, shadows and other atmospheric effects. In this work we probe the potential of applying a cycle-consistent latent variable deep generative model (DGM) for denoising cloudy Sentinel-2 observations conditioned on the information in cloud penetrating bands. We adapt the recently proposed Fréchet Distance m…
A flexible discriminative image denoiser is introduced in which multi-task learning methods are applied to a densoising FCN based on U-Net. The activations of the U-Net model are modified by affine transforms that are a learned function of conditioning inputs. The learning procedure for multiple noise types and levels involves applying a distribution of noise parameters during training to the conditioning inputs, with the same noise parameters applied to a noise generating layer at the input (similar to the approach taken in a denoising autoencoder). It is shown that this flexible denoising mo…
Volumetric Conditional Score-based Residual Diffusion Model for PET/MR Denoising
Massachusetts General Hospital（麻省总医院）；Harvard Medical School（哈佛医学院）；Zhejiang University（浙江大学）；Tsinghua University（清华大学）；Neuroscience Research Institute（神经科学研究所）；Gachon University of Medicine and Science（嘉泉医科大学）
Counterfactual MRI Data Augmentation using Conditional Denoising Diffusion Generative Models
Instituto Superior Técnico（高等技术学院）；Universidade de Lisboa（里斯本大学）；Digital Surgery LAB（数字外科实验室）；Champalimaud Foundation（尚帕利莫基金会）；Faculdade de Medicina, Universidade de Lisboa（里斯本大学医学院）；Instituto de Telecomunicações（电信研究所）
Conditional Denoising Diffusion Model-Based Robust MR Image Reconstruction from Highly Undersampled Data
Department of Mathematics and Systems Engineering（数学与系统工程系）；Department of Biomedical Engineering & Science（生物医学工程与科学系）；Florida Institute of Technology（佛罗里达理工学院）；A. A. Martinos Center for Biomedical Imaging（A.A. Martinos生物医学成像中心）；Massachusetts General Hospital/Harvard Medical School（麻省总医院/哈佛医学院）；Department of Radiology（放射科）；Embry-Riddle Aeronautical University（艾姆伯-里尔航空大学）
Northwestern Polytechnical University（西北工业大学）；Korea Advanced Institute of Science and Technology (KAIST)（韩国科学技术院）
Diffusion probabilistic models (DPM) have been widely adopted in image-to-image translation to generate high-quality images. Prior attempts at applying the DPM to image super-resolution (SR) have shown that iteratively refining a pure Gaussian noise with a conditional image using a U-Net trained on denoising at various-level noises can help obtain a satisfied high-resolution image for the low-resolution one. To further improve the performance and simplify current DPM-based super-resolution methods, we propose a simple but non-trivial DPM-based super-resolution post-process framework,i.e., cDPM…
A Conditional Denoising Diffusion Probabilistic Model for Radio Interferometric Image Reconstruction
The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；Hong Kong University of Science and Technology（香港科技大学）；Guangzhou University（广州大学）
Compressed Image Generation with Denoising Diffusion Codebook Models
Faculty of Computer Science, Technion -- Israel Institute of Technology, Haifa, Israel（计算机科学系，技术离子理工学院——以色列理工学院，海法，以色列）；Faculty of Electrical and Computer Engineering, Technion -- Israel Institute of Technology, Haifa, Israel（电气与计算机工程系，技术离子理工学院——以色列理工学院，海法，以色列）
SABILab（SABI实验室）；École Polytechnique（巴黎综合理工学院）；Institut Polytechnique de Paris（巴黎理工学院）；Université Paris-Saclay（巴黎萨克雷大学）；CNRS（法国国家科学研究中心）；Laboratoire de mathématiques d’Orsay（奥赛数学实验室）；Laboratoire J. A. Dieudonné（J·A·迪厄多内实验室）；Université Côte d’Azur（蔚蓝海岸大学）；Télécom SudParis（南巴黎电信学院）
We propose a novel self-supervised image blind denoising approach in which two neural networks jointly predict the clean signal and infer the noise distribution. Assuming that the noisy observations are independent conditionally to the signal, the networks can be jointly trained without clean training data. Therefore, our approach is particularly relevant for biomedical image denoising where the noise is difficult to model precisely and clean training data are usually unavailable. Our method significantly outperforms current state-of-the-art self-supervised blind denoising algorithms, on six p…
Google AI（谷歌人工智能）；Chevron Energy Technology Company（雪佛龙能源技术公司）；Quantiphi（Quantiphi公司）
We propose GAN-based image enhancement models for frequency enhancement of 2D and 3D seismic images. Seismic imagery is used to understand and characterize the Earth's subsurface for energy exploration. Because these images often suffer from resolution limitations and noise contamination, our proposed method performs large-scale seismic volume frequency enhancement and denoising. The enhanced images reduce uncertainty and improve decisions about issues, such as optimal well placement, that often rely on low signal-to-noise ratio (SNR) seismic volumes. We explored the impact of adding lithology…
GAN-Based Architecture for Low-dose Computed Tomography Imaging Denoising
College of Life Science and Technology, Huazhong University of Science and Technology（华中科技大学生命科学技术学院）；New York University Shanghai（上海纽约大学）；School of Life Sciences and Biotechnology, Shanghai Jiao Tong University（上海交通大学生命科学技术学院）
