Learning-based image denoising methods have been bounded to situations where well-aligned noisy and clean images are given, or samples are synthesized from predetermined noise models, e.g., Gaussian. While recent generative noise modeling methods aim to simulate the unknown distribution of real-world noise, several limitations still exist. In a practical scenario, a noise generator should learn to simulate the general and complex noise distribution without using paired noisy and clean images. However, since existing methods are constructed on the unrealistic assumption of real-world noise, they tend to generate implausible patterns and cannot express complicated noise maps. Therefore, we introduce a Clean-to-Noisy image generation framework, namely C2N, to imitate complex real-world noise without using any paired examples. We construct the noise generator in C2N accordingly with each component of real-world noise characteristics to express a wide range of noise accurately. Combined with our C2N, conventional denoising CNNs can be trained to outperform existing unsupervised methods on challenging real-world benchmarks by a large margin.
Liaoning Normal University（辽宁师范大学）；Jilin University（吉林大学）
In this paper, an image denoising algorithm is proposed for salt and pepper noise. First, a generative model is built on a patch as a basic unit and then the algorithm locates the image noise within that patch in order to better describe the patch and obtain better subsequent clustering. Second, the algorithm classifies patches using a generative clustering method, thus providing additional similarity information for noise repair and suppressing the interference of noise, abandoning those categories that consist of a smaller number of patches. Finally, the algorithm builds a non-local switchin…
PPFM: Image denoising in photon-counting CT using single-step posterior sampling Poisson flow generative models
KTH Royal Institute of Technology（皇家理工学院）；Karolinska University Hospital（卡罗林斯卡大学医院）；University of Wisconsin School of Medicine and Public Health（威斯康星大学麦迪逊分校医学与公共卫生学院）；GE HealthCare（GE医疗）；Rensselaer Polytechnic Institute（伦斯勒理工学院）
Sungkyunkwan University（成均馆大学）；Yonsei University（延世大学）；Seoul National University（首尔国立大学）
We tackle a challenging blind image denoising problem, in which only single distinct noisy images are available for training a denoiser, and no information about noise is known, except for it being zero-mean, additive, and independent of the clean image. In such a setting, which often occurs in practice, it is not possible to train a denoiser with the standard discriminative training or with the recently developed Noise2Noise (N2N) training; the former requires the underlying clean image for the given noisy image, and the latter requires two independently realized noisy image pair for a clean …
Deep learning-based image denoising approaches have been extensively studied in recent years, prevailing in many public benchmark datasets. However, the stat-of-the-art networks are computationally too expensive to be directly applied on mobile devices. In this work, we propose a light-weight, efficient neural network-based raw image denoiser that runs smoothly on mainstream mobile devices, and produces high quality denoising results. Our key insights are twofold: (1) by measuring and estimating sensor noise level, a smaller network trained on synthetic sensor-specific data can out-perform lar…
This document describes a noise generator that simulates realistic noise found in smartphone cameras. The generator simulates Poissonian-Gaussian noise whose parameters have been estimated on the Smartphone Image Denoising Dataset (SIDD). The generator is available online, and is currently being used in compressed-domain denoising exploration experiments in JPEG AI.
YOND: Practical Blind Raw Image Denoising Free from Camera-Specific Data Dependency
School of Computer Science and Technology, Beijing Institute of Technology（计算机科学与技术学院，北京理工大学）；School of Artificial Intelligence, Beijing Normal University（人工智能学院，北京师范大学）
RELD: Regularization by Latent Diffusion Models for Image Restoration
University of Bologna（博洛尼亚大学）；University of Macerata（马切拉塔大学）；University of Modena and Reggio Emilia（摩德纳大学与雷焦艾米利亚大学）；University of Milan（米兰大学）；Washington University in St. Louis（圣路易斯华盛顿大学）
Classical image denoising methods utilize the non-local self-similarity principle to effectively recover image content from noisy images. Current state-of-the-art methods use deep convolutional neural networks (CNNs) to effectively learn the mapping from noisy to clean images. Deep denoising CNNs manifest a high learning capacity and integrate non-local information owing to the large receptive field yielded by numerous cascade of hidden layers. However, deep networks are also computationally complex and require large data for training. To address these issues, this study draws the focus on the…
Harbin Institute of Technology（哈尔滨工业大学）；The Hong Kong Polytechnic University（香港理工大学）
Due to the fast inference and good performance, discriminative learning methods have been widely studied in image denoising. However, these methods mostly learn a specific model for each noise level, and require multiple models for denoising images with different noise levels. They also lack flexibility to deal with spatially variant noise, limiting their applications in practical denoising. To address these issues, we present a fast and flexible denoising convolutional neural network, namely FFDNet, with a tunable noise level map as the input. The proposed FFDNet works on downsampled sub-imag…
Magnetic resonance imaging (MRI) is a common and life-saving medical imaging technique. However, acquiring high signal-to-noise ratio MRI scans requires long scan times, resulting in increased costs and patient discomfort, and decreased throughput. Thus, there is great interest in denoising MRI scans, especially for the subtype of diffusion MRI scans that are severely SNR-limited. While most prior MRI denoising methods are supervised in nature, acquiring supervised training datasets for the multitude of anatomies, MRI scanners, and scan parameters proves impractical. Here, we propose Denoising…
The potential of using remote sensing imagery for environmental modelling and for providing real time support to humanitarian operations such as hurricane relief efforts is well established. These applications are substantially affected by missing data due to non-structural noise such as clouds, shadows and other atmospheric effects. In this work we probe the potential of applying a cycle-consistent latent variable deep generative model (DGM) for denoising cloudy Sentinel-2 observations conditioned on the information in cloud penetrating bands. We adapt the recently proposed Fréchet Distance m…
Peking University（北京大学）；Massachusetts General Hospital（麻省总医院）；Harvard Medical School（哈佛医学院）
In supervised learning for image denoising, usually the paired clean images and noisy images are collected or synthesised to train a denoising model. L2 norm loss or other distance functions are used as the objective function for training. It often leads to an over-smooth result with less image details. In this paper, we regard the denoising task as a problem of estimating the posterior distribution of clean images conditioned on noisy images. We apply the idea of diffusion model to realize generative image denoising. According to the noise model in denoising tasks, we redefine the diffusion p…
Universidad Politécnica de Madrid（马德里理工大学）；Centre national de la recherche scientifique (CNRS)（法国国家科学研究中心）；Interdisciplinary Institute for Neuroscience (IINS)（跨学科神经科学研究所）；University of Bordeaux（波尔多大学）；Mechanobiology Institute（机械生物学研究所）；National University of Singapore（新加坡国立大学）
Microscopy images often suffer from high levels of noise, which can hinder further analysis and interpretation. Content-aware image restoration (CARE) methods have been proposed to address this issue, but they often require large amounts of training data and suffer from over-fitting. To overcome these challenges, we propose a novel framework for few-shot microscopy image denoising. Our approach combines a generative adversarial network (GAN) trained via contrastive learning (CL) with two structure preserving loss terms (Structural Similarity Index and Total Variation loss) to further improve t…
Denoising Diffusion models are gaining increasing popularity in the field of generative modeling for several reasons, including the simple and stable training, the excellent generative quality, and the solid probabilistic foundation. In this article, we address the problem of {\em embedding} an image into the latent space of Denoising Diffusion Models, that is finding a suitable ``noisy'' image whose denoising results in the original image. We particularly focus on Denoising Diffusion Implicit Models due to the deterministic nature of their reverse diffusion process. As a side result of our in…
Counterfactual MRI Data Augmentation using Conditional Denoising Diffusion Generative Models
Instituto Superior Técnico（高等技术学院）；Universidade de Lisboa（里斯本大学）；Digital Surgery LAB（数字外科实验室）；Champalimaud Foundation（尚帕利莫基金会）；Faculdade de Medicina, Universidade de Lisboa（里斯本大学医学院）；Instituto de Telecomunicações（电信研究所）
Generative adversarial networks (GANs) transform low-dimensional latent vectors into visually plausible images. If the real dataset contains only clean images, then ostensibly, the manifold learned by the GAN should contain only clean images. In this paper, we propose to denoise corrupted images by finding the nearest point on the GAN manifold, recovering latent vectors by minimizing distances in image space. We first demonstrate that given a corrupted version of an image that truly lies on the GAN manifold, we can approximately recover the latent vector and denoise the image, obtaining signif…
Ambient Denoising Diffusion Generative Adversarial Networks for Establishing Stochastic Object Models from Noisy Image Data
Global Institute of Future Technology, Shanghai Jiao Tong University（上海交通大学未来技术研究院）；University of Michigan-Shanghai Jiao Tong University Joint Institute, Shanghai Jiao Tong University（上海交通大学密歇根大学联合学院）；University of Arizona（亚利桑那大学）；Wyant College of Optical Sciences, University of Arizona（亚利桑那大学怀恩特光学科学学院）
Rensselaer Polytechnic Institute（伦斯勒理工学院）；University of Massachusetts Lowell（马萨诸塞大学洛厄尔分校）；Xian Jiaotong University（西安交通大学）；Massachusetts General Hospital（麻省总医院）；Harvard Medical School（哈佛医学院）；West China Hospital（华西医院）
In this paper, we introduce a new CT image denoising method based on the generative adversarial network (GAN) with Wasserstein distance and perceptual similarity. The Wasserstein distance is a key concept of the optimal transform theory, and promises to improve the performance of the GAN. The perceptual loss compares the perceptual features of a denoised output against those of the ground truth in an established feature space, while the GAN helps migrate the data noise distribution from strong to weak. Therefore, our proposed method transfers our knowledge of visual perception to the image den…
College of Information Science and Engineering（信息科学与工程学院）；Ocean University of China（中国海洋大学）
Semantic layouts based Image synthesizing, which has benefited from the success of Generative Adversarial Network (GAN), has drawn much attention in these days. How to enhance the synthesis image equality while keeping the stochasticity of the GAN is still a challenge. We propose a novel denoising framework to handle this problem. The overlapped objects generation is another challenging task when synthesizing images from a semantic layout to a realistic RGB photo. To overcome this deficiency, we include a one-hot semantic label map to force the generator paying more attention on the overlapped…
CSEM（瑞士电子与微技术中心）；University of Maryland（马里兰大学）；ETHZ（苏黎世联邦理工学院）
Learning probabilistic models that can estimate the density of a given set of samples, and generate samples from that density, is one of the fundamental challenges in unsupervised machine learning. We introduce a new generative model based on denoising density estimators (DDEs), which are scalar functions parameterized by neural networks, that are efficiently trained to represent kernel density estimators of the data. Leveraging DDEs, our main contribution is a novel technique to obtain generative models by minimizing the KL-divergence directly. We prove that our algorithm for obtaining genera…
IIIT Sri City（斯里城印度信息技术学院）；CSIR-CEERI Pilani（科学与工业研究理事会-中央电子工程研究所皮拉尼）；CCS Computers Pvt Ltd（CCS计算机私人有限公司）；IIT Delhi（印度理工学院德里分校）；IIT Jodhpur（印度理工学院焦特布尔分校）
Synthesizing high quality saliency maps from noisy images is a challenging problem in computer vision and has many practical applications. Samples generated by existing techniques for saliency detection cannot handle the noise perturbations smoothly and fail to delineate the salient objects present in the given scene. In this paper, we present a novel end-to-end coupled Denoising based Saliency Prediction with Generative Adversarial Network (DSAL-GAN) framework to address the problem of salient object detection in noisy images. DSAL-GAN consists of two generative adversarial-networks (GAN) tra…
Low Dose Computed Tomography (LDCT) has offered tremendous benefits in radiation restricted applications, but the quantum noise as resulted by the insufficient number of photons could potentially harm the diagnostic performance. Current image-based denoising methods tend to produce a blur effect on the final reconstructed results especially in high noise levels. In this paper, a deep learning based approach was proposed to mitigate this problem. An adversarially trained network and a sharpness detection network were trained to guide the training process. Experiments on both simulated and real …
National Institute for Mathematical Sciences（国家数学科学研究所）；Chungnam National University College of Medicine（忠南大学医学院）；Chungnam National University Hospital（忠南大学医院）；Tongji University（同济大学）；Yonsei University（延世大学）
This paper proposes a deep learning-based denoising method for noisy low-dose computerized tomography (CT) images in the absence of paired training data. The proposed method uses a fidelity-embedded generative adversarial network (GAN) to learn a denoising function from unpaired training data of low-dose CT (LDCT) and standard-dose CT (SDCT) images, where the denoising function is the optimal generator in the GAN framework. This paper analyzes the f-GAN objective to derive a suitable generator that is optimized by minimizing a weighted sum of two losses: the Kullback-Leibler divergence between…
Seoul National University（首尔大学）；Samsung SDS（三星SDS）
Denoising diffusion probabilistic models (DDPM) have shown remarkable performance in unconditional image generation. However, due to the stochasticity of the generative process in DDPM, it is challenging to generate images with the desired semantics. In this work, we propose Iterative Latent Variable Refinement (ILVR), a method to guide the generative process in DDPM to generate high-quality images based on a given reference image. Here, the refinement of the generative process in DDPM enables a single DDPM to sample images from various sets directed by the reference image. The proposed ILVR m…
Certain facial parts are salient (unique) in appearance, which substantially contribute to the holistic recognition of a subject. Occlusion of these salient parts deteriorates the performance of face recognition algorithms. In this paper, we propose a generative model to reconstruct the missing parts of the face which are under occlusion. The proposed generative model (SD-GAN) reconstructs a face preserving the illumination variation and identity of the face. A novel adversarial training algorithm has been designed for a bimodal mutually exclusive Generative Adversarial Network (GAN) model, fo…
Fudan University（复旦大学）；Sichuan University（四川大学）；Shanghai Center for Brain Science and Brain-inspired Technology（上海脑科学与类脑研究中心）
LDCT has drawn major attention in the medical imaging field due to the potential health risks of CT-associated X-ray radiation to patients. Reducing the radiation dose, however, decreases the quality of the reconstructed images, which consequently compromises the diagnostic performance. Various deep learning techniques have been introduced to improve the image quality of LDCT images through denoising. GANs-based denoising methods usually leverage an additional classification network, i.e. discriminator, to learn the most discriminate difference between the denoised and normal-dose images and, …
Peking University（北京大学）；Academy for Advanced Interdisciplinary Studies（前沿交叉学科研究院）；Massachusetts General Hospital（麻省总医院）；Harvard Medical School（哈佛医学院）
We propose a novel and unified method, measurement-conditioned denoising diffusion probabilistic model (MC-DDPM), for under-sampled medical image reconstruction based on DDPM. Different from previous works, MC-DDPM is defined in measurement domain (e.g. k-space in MRI reconstruction) and conditioned on under-sampling mask. We apply this method to accelerate MRI reconstruction and the experimental results show excellent performance, outperforming full supervision baseline and the state-of-the-art score-based reconstruction method. Due to its generative nature, MC-DDPM can also quantify the unce…
Sichuan University（四川大学）；Chengdu University of Information Technology（成都信息工程大学）；Southeast University（东南大学）；West China Hospital of Sichuan University（四川大学华西医院）；Southern Medical University（南方医科大学）
Structure-preserved denoising of 3D magnetic resonance imaging (MRI) images is a critical step in medical image analysis. Over the past few years, many algorithms with impressive performances have been proposed. In this paper, inspired by the idea of deep learning, we introduce an MRI denoising method based on the residual encoder-decoder Wasserstein generative adversarial network (RED-WGAN). Specifically, to explore the structure similarity between neighboring slices, a 3D configuration is utilized as the basic processing unit. Residual autoencoders combined with deconvolution operations are …
Mila（米拉研究所）；University of Montreal（蒙特利尔大学）；McGill University（麦吉尔大学）；Polytechnique Montréal（蒙特利尔理工学院）；Service Now（ServiceNow公司）
Generative models based on denoising diffusion techniques have led to an unprecedented increase in the quality and diversity of imagery that is now possible to create with neural generative models. However, most contemporary state-of-the-art methods are derived from a standard isotropic Gaussian formulation. In this work we examine the situation where non-isotropic Gaussian distributions are used. We present the key mathematical derivations for creating denoising diffusion models using an underlying non-isotropic Gaussian noise model. We also provide initial experiments with the CIFAR-10 datas…
The Chinese University of Hong Kong（香港中文大学）；SmartMore（思谋科技）；Sensetime Research（商汤科技研究院）；Tetras AI（Tetras人工智能）
To facilitate video denoising research, we construct a compelling dataset, namely, "Practical Video Denoising Dataset" (PVDD), containing 200 noisy-clean dynamic video pairs in both sRGB and RAW format. Compared with existing datasets consisting of limited motion information, PVDD covers dynamic scenes with varying and natural motion. Different from datasets using primarily Gaussian or Poisson distributions to synthesize noise in the sRGB domain, PVDD synthesizes realistic noise from the RAW domain with a physically meaningful sensor noise model followed by ISP processing. Moreover, we also pr…
Diffusion models are powerful generative models that simulate the reverse of diffusion processes using score functions to synthesize data from noise. The sampling process of diffusion models can be interpreted as solving the reverse stochastic differential equation (SDE) or the ordinary differential equation (ODE) of the diffusion process, which often requires up to thousands of discretization steps to generate a single image. This has sparked a great interest in developing efficient integration techniques for reverse-S/ODEs. Here, we propose an orthogonal approach to accelerating score-based …
C-DiffDet+: Fusing Global Scene Context with Generative Denoising for High-Fidelity Car Damage Detection
organization= Department of Innovation Engineering, University of Salento \& Institute of Applied Sciences；organization= Electrical Engineering Department , addressline= Mohamed Khider University of Biskra , postcode= 07000 , city= Biskra , country= Algeria；organization= UPV/EHU , addressline= University of the Basque Country , city= Sebastian , postcode= 20018 , state= Sebastian , country= Spain；organization= Sorbonne University Abu Dhabi , addressline= Al Reem Island, P.O. Box 34105 , city= Abu Dhabi , postcode= 34105 , state= Abu Dhabi , country= United Arab Emirates；organization= CNR-STIIMA, Institute of Intelligent Industrial Systems；Technologies for Advanced Manufacturing , addressline= c/o Campus Ecotekne, Via Monteroni , city= Lecce , postcode= 73100 , state= LE , country= Italy；organization= Institute of Applied Sciences
Denoising Vision Transformer Autoencoder with Spectral Self-Regularization
Nanjing University（南京大学）；Kling Team, Kuaishou Technology（快手科技 Kling 团队）；Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））；Shanghai Jiao Tong University（上海交通大学）；University of Science and Technology of China（中国科学技术大学）
Tel-Aviv University（特拉维夫大学）；Facebook AI Research（脸书人工智能研究）；École Normale Supérieure Paris-Saclay（巴黎-萨克兰高等师范学院）
Generative diffusion processes are an emerging and effective tool for image and speech generation. In the existing methods, the underlying noise distribution of the diffusion process is Gaussian noise. However, fitting distributions with more degrees of freedom could improve the performance of such generative models. In this work, we investigate other types of noise distribution for the diffusion process. Specifically, we introduce the Denoising Diffusion Gamma Model (DDGM) and show that noise from Gamma distribution provides improved results for image and speech generation. Our approach prese…
DDFM: Denoising Diffusion Model for Multi-Modality Image Fusion
Xi’an Jiaotong University（西安交通大学）；Computer Vision Lab, ETH Zürich（苏黎世联邦理工学院计算机视觉实验室）；Northwestern Polytechnical University（西北工业大学）；Macau University of Science and Technology（澳门科技大学）；University of Würzburg（维尔茨堡大学）
