Today, Convolutional Neural Networks (CNNs) are the leading method for image denoising. They are traditionally trained on pairs of images, which are often hard to obtain for practical applications. This motivates self-supervised training methods such as Noise2Void~(N2V) that operate on single noisy images. Self-supervised methods are, unfortunately, not competitive with models trained on image pairs. Here, we present 'Probabilistic Noise2Void' (PN2V), a method to train CNNs to predict per-pixel intensity distributions. Combining these with a suitable description of the noise, we obtain a complete probabilistic model for the noisy observations and true signal in every pixel. We evaluate PN2V on publicly available microscopy datasets, under a broad range of noise regimes, and achieve competitive results with respect to supervised state-of-the-art methods.
Supervised learning-based methods yield robust denoising results, yet they are inherently limited by the need for large-scale clean/noisy paired datasets. The use of unsupervised denoisers, on the other hand, necessitates a more detailed understanding of the underlying image statistics. In particular, it is well known that apparent differences between clean and noisy images are most prominent on high-frequency bands, justifying the use of low-pass filters as part of conventional image preprocessing steps. However, most learning-based denoising methods utilize only one-sided information from th…
University of Electronic Science and Technology of China（电子科技大学）；Oregon State University（俄勒冈州立大学）；RIKEN Center for Advanced Intelligence Project（理化学研究所先进智能项目中心）
Image denoising is often empowered by accurate prior information. In recent years, data-driven neural network priors have shown promising performance for RGB natural image denoising. Compared to classic handcrafted priors (e.g., sparsity and total variation), the "deep priors" are learned using a large number of training samples -- which can accurately model the complex image generating process. However, data-driven priors are hard to acquire for hyperspectral images (HSIs) due to the lack of training data. A remedy is to use the so-called unsupervised deep image prior (DIP). Under the unsuper…
Rician Denoising Diffusion Probabilistic Models For Sodium Breast MRI Enhancement
University of Cambridge（剑桥大学）；Universitat de Barcelona（巴塞罗那大学）；Haukeland University Hospital（豪克兰大学医院）；University of York（约克大学）；Manchester Metropolitan University（曼彻斯特城市大学）
Image denoising is a prerequisite for downstream tasks in many fields. Low-dose and photon-counting computed tomography (CT) denoising can optimize diagnostic performance at minimized radiation dose. Supervised deep denoising methods are popular but require paired clean or noisy samples that are often unavailable in practice. Limited by the independent noise assumption, current unsupervised denoising methods cannot process correlated noises as in CT images. Here we propose the first-of-its-kind similarity-based unsupervised deep denoising approach, referred to as Noise2Sim, that works in a non…
Google Research（谷歌研究院）；Fondazione Human Technopole（人类技术城基金会）
Image denoising and artefact removal are complex inverse problems admitting multiple valid solutions. Unsupervised diversity restoration, that is, obtaining a diverse set of possible restorations given a corrupted image, is important for ambiguity removal in many applications such as microscopy where paired data for supervised training are often unobtainable. In real world applications, imaging noise and artefacts are typically hard to model, leading to unsatisfactory performance of existing unsupervised approaches. This work presents an interpretable approach for unsupervised and diverse imag…
This paper evaluates the performance of supervised and unsupervised deep learning models for denoising positron emission tomography (PET) images in the presence of reduced acquisition times. Our experiments consider 212 studies (56908 images), and evaluate the models using 2D (RMSE, SSIM) and 3D (SUVpeak and SUVmax error for the regions of interest) metrics. It was shown that, in contrast to previous studies, supervised models (ResNet, Unet, SwinIR) outperform unsupervised models (pix2pix GAN and CycleGAN with ResNet backbone and various auxiliary losses) in the reconstruction of 2D PET images…
With the advent of recent advances in unsupervised learning, efficient training of a deep network for image denoising without pairs of noisy and clean images has become feasible. However, most current unsupervised denoising methods are built on the assumption of zero-mean noise under the signal-independent condition. This assumption causes blind denoising techniques to suffer brightness shifting problems on images that are greatly corrupted by extreme noise such as salt-and-pepper noise. Moreover, most blind denoising methods require a random masking scheme for training to ensure the invarianc…
Gordon Center for Medical Imaging, Massachusetts General Hospital, Harvard Medical School（哈佛医学院麻省总医院戈登医学成像中心）；Center for Advanced Medical Computing and Analysis, Massachusetts General Hospital, Harvard Medical School（哈佛医学院麻省总医院高级医学计算与分析中心）；University of Texas MD Anderson Cancer Center（德克萨斯大学MD安德森癌症中心）
Due to various physical degradation factors and limited counts received, PET image quality needs further improvements. The denoising diffusion probabilistic models (DDPM) are distribution learning-based models, which try to transform a normal distribution into a specific data distribution based on iterative refinements. In this work, we proposed and evaluated different DDPM-based methods for PET image denoising. Under the DDPM framework, one way to perform PET image denoising is to provide the PET image and/or the prior image as the network input. Another way is to supply the prior image as th…
The Chinese University of Hong Kong（香港中文大学）；SenseTime Research（商汤科技研究院）
The lack of large-scale noisy-clean image pairs restricts supervised denoising methods' deployment in actual applications. While existing unsupervised methods are able to learn image denoising without ground-truth clean images, they either show poor performance or work under impractical settings (e.g., paired noisy images). In this paper, we present a practical unsupervised image denoising method to achieve state-of-the-art denoising performance. Our method only requires single noisy images and a noise model, which is easily accessible in practical raw image denoising. It performs two steps it…
Politecnico di Milano（米兰理工大学）；University of Twente（特温特大学）；University of Bath（巴斯大学）；University of Cambridge（剑桥大学）
4D flow MRI is a non-invasive imaging method that can measure blood flow velocities over time. However, the velocity fields detected by this technique have limitations due to low resolution and measurement noise. Coordinate-based neural networks have been researched to improve accuracy, with SIRENs being suitable for super-resolution tasks. Our study investigates SIRENs for time-varying 3-directional velocity fields measured in the aorta by 4D flow MRI, achieving denoising and super-resolution. We trained our method on voxel coordinates and benchmarked our approach using synthetic measurements…
Many interesting tasks in image restoration can be cast as linear inverse problems. A recent family of approaches for solving these problems uses stochastic algorithms that sample from the posterior distribution of natural images given the measurements. However, efficient solutions often require problem-specific supervised training to model the posterior, whereas unsupervised methods that are not problem-specific typically rely on inefficient iterative methods. This work addresses these issues by introducing Denoising Diffusion Restoration Models (DDRM), an efficient, unsupervised posterior sa…
Technion Institute of Technology（以色列理工学院）；Google Research（谷歌研究院）
Image denoising is a well studied problem with an extensive activity that has spread over several decades. Despite the many available denoising algorithms, the quest for simple, powerful and fast denoisers is still an active and vibrant topic of research. Leading classical denoising methods are typically designed to exploit the inner structure in images by modeling local overlapping patches, while operating in an unsupervised fashion. In contrast, recent newcomers to this arena are supervised and universal neural-network-based methods that bypass this modeling altogether, targeting the inferen…
Hamamatsu Photonics K. K.（滨松光子学株式会社）；Hamamatsu University School of Medicine（滨松医科大学）
Although supervised convolutional neural networks (CNNs) often outperform conventional alternatives for denoising positron emission tomography (PET) images, they require many low- and high-quality reference PET image pairs. Herein, we propose an unsupervised 3D PET image denoising method based on an anatomical information-guided attention mechanism. The proposed magnetic resonance-guided deep decoder (MR-GDD) utilizes the spatial details and semantic features of MR-guidance image more effectively by introducing encoder-decoder and deep decoder subnetworks. Moreover, the specific shapes and pat…
Center for Systems Biology Dresden (CSBD)（德累斯顿系统生物学中心（CSBD））；Max-Planck Institute of Molecular Cell Biology and Genetics（马克斯·普朗克分子细胞生物学与遗传学研究所）
Image denoising is the first step in many biomedical image analysis pipelines and Deep Learning (DL) based methods are currently best performing. A new category of DL methods such as Noise2Void or Noise2Self can be used fully unsupervised, requiring nothing but the noisy data. However, this comes at the price of reduced reconstruction quality. The recently proposed Probabilistic Noise2Void (PN2V) improves results, but requires an additional noise model for which calibration data needs to be acquired. Here, we present improvements to PN2V that (i) replace histogram based noise models by paramet…
Peking University（北京大学）；Academy for Advanced Interdisciplinary Studies（前沿交叉学科研究院）；Massachusetts General Hospital（麻省总医院）；Harvard Medical School（哈佛医学院）
We propose a novel and unified method, measurement-conditioned denoising diffusion probabilistic model (MC-DDPM), for under-sampled medical image reconstruction based on DDPM. Different from previous works, MC-DDPM is defined in measurement domain (e.g. k-space in MRI reconstruction) and conditioned on under-sampling mask. We apply this method to accelerate MRI reconstruction and the experimental results show excellent performance, outperforming full supervision baseline and the state-of-the-art score-based reconstruction method. Due to its generative nature, MC-DDPM can also quantify the unce…
Korea Advanced Institute of Science and Technology (KAIST)（韩国科学技术院（KAIST））；VUNO Corp.（VUNO公司）；Chungnam National University College of Medicine（忠南大学医学院）；Chungnam National University Hospital（忠南大学医院）
The acquisition conditions for low-dose and high-dose CT images are usually different, so that the shifts in the CT numbers often occur. Accordingly, unsupervised deep learning-based approaches, which learn the target image distribution, often introduce CT number distortions and result in detrimental effects in diagnostic performance. To address this, here we propose a novel unsupervised learning approach for lowdose CT reconstruction using patch-wise deep metric learning. The key idea is to learn embedding space by pulling the positive pairs of image patches which shares the same anatomical s…
We propose a novel unpaired image-to-image translation method that uses denoising diffusion probabilistic models without requiring adversarial training. Our method, UNpaired Image Translation with Denoising Diffusion Probabilistic Models (UNIT-DDPM), trains a generative model to infer the joint distribution of images over both domains as a Markov chain by minimising a denoising score matching objective conditioned on the other domain. In particular, we update both domain translation models simultaneously, and we generate target domain images by a denoising Markov Chain Monte Carlo approach tha…
Recently, much progress has been made in unsupervised denoising learning. However, existing methods more or less rely on some assumptions on the signal and/or degradation model, which limits their practical performance. How to construct an optimal criterion for unsupervised denoising learning without any prior knowledge on the degradation model is still an open question. Toward answering this question, this work proposes a criterion for unsupervised denoising learning based on the optimal transport theory. This criterion has favorable properties, e.g., approximately maximal preservation of the…
Center for Systems Biology Dresden（德累斯顿系统生物学中心）；Max-Planck Institute (CBG)（马克斯·普朗克分子细胞生物学与遗传学研究所）；School of Computer Science（计算机科学学院）；University of Birmingham（伯明翰大学）；Fondazione Human Technopole（人类技术基金会）
Deep Learning based methods have emerged as the indisputable leaders for virtually all image restoration tasks. Especially in the domain of microscopy images, various content-aware image restoration (CARE) approaches are now used to improve the interpretability of acquired data. Naturally, there are limitations to what can be restored in corrupted images, and like for all inverse problems, many potential solutions exist, and one of them must be chosen. Here, we propose DivNoising, a denoising approach based on fully convolutional variational autoencoders (VAEs), overcoming the problem of havin…
Nosie is an important cause of low quality Optical coherence tomography (OCT) image. The neural network model based on Convolutional neural networks(CNNs) has demonstrated its excellent performance in image denoising. However, OCT image denoising still faces great challenges because many previous neural network algorithms required a large number of labeled data, which might cost much time or is expensive. Besides, these CNN-based algorithms need numerous parameters and good tuning techniques, which is hardware resources consuming. To solved above problems, We proposed a new Cycle-Consistent Ge…
University of Texas Southwestern Medical Centre（德克萨斯大学西南医学中心）
Despite the indispensable role of X-ray computed tomography (CT) in diagnostic medicine field, the associated ionizing radiation is still a major concern considering that it may cause genetic and cancerous diseases. Decreasing the exposure can reduce the dose and hence the radiation-related risk, but will also induce higher quantum noise. Supervised deep learning can be used to train a neural network to denoise the low-dose CT (LDCT). However, its success requires massive pixel-wise paired LDCT and normal-dose CT (NDCT) images, which are rarely available in real practice. To alleviate this pro…
Indian Institute of Technology Madras（印度理工学院马德拉斯分校）；Robert Bosch Center for Data Science and AI（罗伯特·博世数据科学与人工智能中心）
Deep convolutional neural networks (CNNs) for video denoising are typically trained with supervision, assuming the availability of clean videos. However, in many applications, such as microscopy, noiseless videos are not available. To address this, we propose an Unsupervised Deep Video Denoiser (UDVD), a CNN architecture designed to be trained exclusively with noisy data. The performance of UDVD is comparable to the supervised state-of-the-art, even when trained only on a single short noisy video. We demonstrate the promise of our approach in real-world imaging applications by denoising raw vi…
Diffusion models have shown promising results on single-image super-resolution and other image- to-image translation tasks. Despite this success, they have not outperformed state-of-the-art GAN models on the more challenging blind super-resolution task, where the input images are out of distribution, with unknown degradations. This paper introduces SR3+, a diffusion-based model for blind super-resolution, establishing a new state-of-the-art. To this end, we advocate self-supervised training with a combination of composite, parameterized degradations for self-supervised training, and noise-cond…
Korea Advanced Institute of Science and Technology (KAIST)（韩国科学技术院）；Korea Aerospace Research Institute (KARI)（韩国航空宇宙研究院）
Multi-spectral satellite imaging sensors acquire various spectral band images such as red (R), green (G), blue (B), near-infrared (N), etc. Thanks to the unique spectroscopic property of each spectral band with respective to the objects on the ground, multi-spectral satellite imagery can be used for various geological survey applications. Unfortunately, image artifacts from imaging sensor noises often affect the quality of scenes and have negative impacts on the applications of satellite imagery. Recently, deep learning approaches have been extensively explored for the removal of noises in sat…
University of Oulu（奥卢大学）；University College London（伦敦大学学院）
Multi-energy computed tomography (CT) with photon counting detectors (PCDs) enables spectral imaging as PCDs can assign the incoming photons to specific energy channels. However, PCDs with many spectral channels drastically increase the computational complexity of the CT reconstruction, and bespoke reconstruction algorithms need fine-tuning to varying noise statistics. \rev{Especially if many projections are taken, a large amount of data has to be collected and stored. Sparse view CT is one solution for data reduction. However, these issues are especially exacerbated when sparse imaging scenar…
B-mode ultrasound tongue imaging is widely used in the speech production field. However, efficient interpretation is in a great need for the tongue image sequences. Inspired by the recent success of unsupervised deep learning approach, we explore unsupervised convolutional network architecture for the feature extraction in the ultrasound tongue image, which can be helpful for the clinical linguist and phonetics. By quantitative comparison between different unsupervised feature extraction approaches, the denoising convolutional autoencoder (DCAE)-based method outperforms the other feature extra…
Korea Advanced Institute of Science and Technology (KAIST)（韩国科学技术院）
Recently, CycleGAN was shown to provide high-performance, ultra-fast denoising for low-dose X-ray computed tomography (CT) without the need for a paired training dataset. Although this was possible thanks to cycle consistency, CycleGAN requires two generators and two discriminators to enforce cycle consistency, demanding significant GPU resources and technical skills for training. A recent proposal of tunable CycleGAN with Adaptive Instance Normalization (AdaIN) alleviates the problem in part by using a single generator. However, two discriminators and an additional AdaIN code generator are st…
Low-dose computed tomography (LDCT) is an important topic in the field of radiology over the past decades. LDCT reduces ionizing radiation-induced patient health risks but it also results in a low signal-to-noise ratio (SNR) and a potential compromise in the diagnostic performance. In this paper, to improve the LDCT denoising performance, we introduce the conditional denoising diffusion probabilistic model (DDPM) and show encouraging results with a high computational efficiency. Specifically, given the high sampling cost of the original DDPM model, we adapt the fast ordinary differential equat…
Korea Advanced Institute of Science and Technology（韩国科学技术院）
Recently, deep learning approaches have been extensively studied for low-dose CT denoising thanks to its superior performance despite the fast computational time. In particular, cycleGAN has been demonstrated as a powerful unsupervised learning scheme to improve the low-dose CT image quality without requiring matched high-dose reference data. Unfortunately, one of the main limitations of the cycleGAN approach is that it requires two deep neural network generators at the training phase, although only one of them is used at the inference phase. The secondary auxiliary generator is needed to enfo…
