Adaptive Semantic-Enhanced Denoising Diffusion Probabilistic Model for Remote Sensing Image Super-Resolution
School of Science and Engineering, The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳）理工学院）；School of Information Science and Engineering, Wuhan University of Science and Technology（武汉科技大学信息科学与工程学院）；College of Urban and Environment Sciences, Central China Normal University（华中师范大学城市与环境科学学院）
Jialu Sui, Xianping Ma, Xiaokang Zhang, Man-On Pun
Remote sensing image super-resolution (SR) is a crucial task to restore high-resolution (HR) images from low-resolution (LR) observations. Recently, the Denoising Diffusion Probabilistic Model (DDPM) has shown promising performance in image reconstructions by overcoming problems inherent in generative models, such as over-smoothing and mode collapse. However, the high-frequency details generated by DDPM often suffer from misalignment with HR images due to the model's tendency to overlook long-range semantic contexts. This is attributed to the widely used U-Net decoder in the conditional noise predictor, which tends to overemphasize local information, leading to the generation of noises with significant variances during the prediction process. To address these issues, an adaptive semantic-enhanced DDPM (ASDDPM) is proposed to enhance the detail-preserving capability of the DDPM by incorporating low-frequency semantic information provided by the Transformer. Specifically, a novel adaptive diffusion Transformer decoder (ADTD) is developed to bridge the semantic gap between the encoder and decoder through regulating the noise prediction with the global contextual relationships and long-range dependencies in the diffusion process. Additionally, a residual feature fusion strategy establishes information exchange between the two decoders at multiple levels. As a result, the predicted noise generated by our approach closely approximates that of the real noise distribution.Extensive experiments on two SR and two semantic segmentation datasets confirm the superior performance of the proposed ASDDPM in both SR and the subsequent downstream applications. The source code will be available at https://github.com/littlebeen/ASDDPM-Adaptive-Semantic-Enhanced-DDPM.
Most raw data download from satellites are useless, resulting in transmission waste, one solution is to process data directly on satellites, then only transmit the processed results to the ground. Image processing is the main data processing on satellites, in this paper, we focus on image denoising which is the basic image processing. There are many high-performance denoising approaches at present, however, most of them rely on advanced computing resources or rich images on the ground. Considering the limited computing resources of satellites and the characteristics of remote sensing images, w…
Rician Denoising Diffusion Probabilistic Models For Sodium Breast MRI Enhancement
University of Cambridge（剑桥大学）；Universitat de Barcelona（巴塞罗那大学）；Haukeland University Hospital（豪克兰大学医院）；University of York（约克大学）；Manchester Metropolitan University（曼彻斯特城市大学）
MPI-CBG/PKS (CSBD)（马克斯·普朗克分子细胞生物学与遗传学研究所/普朗克计算生物学中心（德累斯顿系统生物学中心））；Brno University of Technology（布尔诺理工大学）
Today, Convolutional Neural Networks (CNNs) are the leading method for image denoising. They are traditionally trained on pairs of images, which are often hard to obtain for practical applications. This motivates self-supervised training methods such as Noise2Void~(N2V) that operate on single noisy images. Self-supervised methods are, unfortunately, not competitive with models trained on image pairs. Here, we present 'Probabilistic Noise2Void' (PN2V), a method to train CNNs to predict per-pixel intensity distributions. Combining these with a suitable description of the noise, we obtain a compl…
Gordon Center for Medical Imaging, Massachusetts General Hospital, Harvard Medical School（哈佛医学院麻省总医院戈登医学成像中心）；Center for Advanced Medical Computing and Analysis, Massachusetts General Hospital, Harvard Medical School（哈佛医学院麻省总医院高级医学计算与分析中心）；University of Texas MD Anderson Cancer Center（德克萨斯大学MD安德森癌症中心）
Due to various physical degradation factors and limited counts received, PET image quality needs further improvements. The denoising diffusion probabilistic models (DDPM) are distribution learning-based models, which try to transform a normal distribution into a specific data distribution based on iterative refinements. In this work, we proposed and evaluated different DDPM-based methods for PET image denoising. Under the DDPM framework, one way to perform PET image denoising is to provide the PET image and/or the prior image as the network input. Another way is to supply the prior image as th…
EDiffSR: An Efficient Diffusion Probabilistic Model for Remote Sensing Image Super-Resolution
Wuhan University（武汉大学）；Harbin Institute of Technology（哈尔滨工业大学）；State Key Laboratory of Information Engineering in Surveying, Mapping, and Remote Sensing, Wuhan University（武汉大学测绘遥感信息工程国家重点实验室）
DiffCMR: Fast Cardiac MRI Reconstruction with Diffusion Probabilistic Models
The Hong Kong University of Science and Technology（香港科技大学）；Southern University of Science and Technology（南方科技大学）；HKUST Shenzhen Research Institute（香港科技大学深圳研究院）
DifFIQA: Face Image Quality Assessment Using Denoising Diffusion Probabilistic Models
University of Ljubljana, Faculty of Electrical Engineering（卢布尔雅那大学电气工程学院）；University of Ljubljana, Faculty of Computer and Information Science（卢布尔雅那大学计算机与信息科学学院）
Resfusion: Denoising Diffusion Probabilistic Models for Image Restoration Based on Prior Residual Noise
College of Computer Science, Nankai University（南开大学计算机学院）；School of Statistics and Data Science, Nankai University（南开大学统计与数据科学学院）；Haihe Lab of ITAI（海河人工智能研究院）；Institute of High Performance Computing, A*STAR（新加坡科技研究局高性能计算研究所）
Regularization by denoising: Bayesian model and Langevin-within-split Gibbs sampling
Institut Denis Poisson, University of Orléans（德尼·泊松研究所，奥尔良大学）；University of Toulouse, IRIT/INP-ENSEEIHT, CNRS（图卢兹大学，法国国家信息与自动化研究所/国立高等电子、电子技术、计算机与机械学院，法国国家科学研究中心）
本文提出一种基于去噪正则化（RED）的图像反演贝叶斯框架，并设计了内含Langevin Monte Carlo步骤的分裂Gibbs采样算法，通过去模糊等任务验证了其有效性。
Image denoising based on a probabilistic model of local image patches has been employed by various researchers, and recently a deep (denoising) autoencoder has been proposed by Burger et al. [2012] and Xie et al. [2012] as a good model for this. In this paper, we propose that another popular family of models in the field of deep learning, called Boltzmann machines, can perform image denoising as well as, or in certain cases of high level of noise, better than denoising autoencoders. We empirically evaluate the two models on three different sets of images with different types and levels of nois…
The potential of using remote sensing imagery for environmental modelling and for providing real time support to humanitarian operations such as hurricane relief efforts is well established. These applications are substantially affected by missing data due to non-structural noise such as clouds, shadows and other atmospheric effects. In this work we probe the potential of applying a cycle-consistent latent variable deep generative model (DGM) for denoising cloudy Sentinel-2 observations conditioned on the information in cloud penetrating bands. We adapt the recently proposed Fréchet Distance m…
Peking University（北京大学）；Academy for Advanced Interdisciplinary Studies（前沿交叉学科研究院）；Massachusetts General Hospital（麻省总医院）；Harvard Medical School（哈佛医学院）
We propose a novel and unified method, measurement-conditioned denoising diffusion probabilistic model (MC-DDPM), for under-sampled medical image reconstruction based on DDPM. Different from previous works, MC-DDPM is defined in measurement domain (e.g. k-space in MRI reconstruction) and conditioned on under-sampling mask. We apply this method to accelerate MRI reconstruction and the experimental results show excellent performance, outperforming full supervision baseline and the state-of-the-art score-based reconstruction method. Due to its generative nature, MC-DDPM can also quantify the unce…
We propose a novel unpaired image-to-image translation method that uses denoising diffusion probabilistic models without requiring adversarial training. Our method, UNpaired Image Translation with Denoising Diffusion Probabilistic Models (UNIT-DDPM), trains a generative model to infer the joint distribution of images over both domains as a Markov chain by minimising a denoising score matching objective conditioned on the other domain. In particular, we update both domain translation models simultaneously, and we generate target domain images by a denoising Markov Chain Monte Carlo approach tha…
University of Texas Southwestern Medical Centre（德克萨斯大学西南医学中心）
Despite the indispensable role of X-ray computed tomography (CT) in diagnostic medicine field, the associated ionizing radiation is still a major concern considering that it may cause genetic and cancerous diseases. Decreasing the exposure can reduce the dose and hence the radiation-related risk, but will also induce higher quantum noise. Supervised deep learning can be used to train a neural network to denoise the low-dose CT (LDCT). However, its success requires massive pixel-wise paired LDCT and normal-dose CT (NDCT) images, which are rarely available in real practice. To alleviate this pro…
Diffusion models have shown promising results on single-image super-resolution and other image- to-image translation tasks. Despite this success, they have not outperformed state-of-the-art GAN models on the more challenging blind super-resolution task, where the input images are out of distribution, with unknown degradations. This paper introduces SR3+, a diffusion-based model for blind super-resolution, establishing a new state-of-the-art. To this end, we advocate self-supervised training with a combination of composite, parameterized degradations for self-supervised training, and noise-cond…
Universal Adversarial Defense in Remote Sensing Based on Pre-trained Denoising Diffusion Models
Helmholtz Institute Freiberg for Resource Technology（亥姆霍兹弗莱贝格资源技术研究所）；Helmholtz-Zentrum Dresden-Rossendorf (HZDR)（德累斯顿-罗森多夫亥姆霍兹中心）；Linköping University（林雪平大学）
Low-dose computed tomography (LDCT) is an important topic in the field of radiology over the past decades. LDCT reduces ionizing radiation-induced patient health risks but it also results in a low signal-to-noise ratio (SNR) and a potential compromise in the diagnostic performance. In this paper, to improve the LDCT denoising performance, we introduce the conditional denoising diffusion probabilistic model (DDPM) and show encouraging results with a high computational efficiency. Specifically, given the high sampling cost of the original DDPM model, we adapt the fast ordinary differential equat…
Denoising Diffusion Probabilistic Models for Image Inpainting of Cell Distributions in the Human Brain
Institute of Neuroscience and Medicine (INM-1), Research Centre Jülich（于利希研究中心神经科学与医学研究所（INM-1））；Helmholtz AI, Research Centre Jülich（于利希研究中心亥姆霍兹人工智能）；Institute of Computer Science, Heinrich-Heine-University Düsseldorf（杜塞尔多夫海因里希·海涅大学计算机科学研究所）
Denoising Plane Wave Ultrasound Images Using Diffusion Probabilistic Models
Concordia University（康考迪亚大学）；Sunnybrook Research Institute（桑尼布鲁克研究所）；INSA-Lyon（里昂国立应用科学学院）；UCBL（里昂第一大学）；CNRS（法国国家科学研究中心）；Inserm（法国国家健康与医学研究院）；Université de Lyon（里昂大学）
Retinal OCT Synthesis with Denoising Diffusion Probabilistic Models for Layer Segmentation
Institute of Imaging and Computer Vision, RWTH Aachen University（成像与计算机视觉研究所，亚琛RWTH大学）；Bio-manufacturing Engineering Laboratory of Tsinghua SIGS, Tsinghua University（清华大学SIGS生物制造工程实验室）；Shenzhen Eye Hospital（深圳眼科医院）；Department of Ophthalmology, RWTH Aachen University（眼科部，亚琛RWTH大学）
University Hospital Aachen（亚琛大学医院）；RWTH Aachen University（亚琛工业大学）；Ocumeda AG（奥库梅达股份公司）；University Hospital Heidelberg（海德堡大学医院）；University Hospital Würzburg（维尔茨堡大学医院）；University of Mainz（美因茨大学）；RWTH Aachen（亚琛工业大学）
Recent advances in computer vision have shown promising results in image generation. Diffusion probabilistic models in particular have generated realistic images from textual input, as demonstrated by DALL-E 2, Imagen and Stable Diffusion. However, their use in medicine, where image data typically comprises three-dimensional volumes, has not been systematically evaluated. Synthetic images may play a crucial role in privacy preserving artificial intelligence and can also be used to augment small datasets. Here we show that diffusion probabilistic models can synthesize high quality medical imagi…
Rice University（莱斯大学）；Microsoft Research AI（微软研究院人工智能）
Attention is a powerful concept in computer vision. End-to-end networks that learn to focus selectively on regions of an image or video often perform strongly. However, other image regions, while not necessarily containing the signal of interest, may contain useful context. We present an approach that exploits the idea that statistics of noise may be shared between the regions that contain the signal of interest and those that do not. Our technique uses the inverse of an attention mask to generate a noise estimate that is then used to denoise temporal observations. We apply this to the task of…
Sparse-view computed tomography (CT) can be used to reduce radiation dose greatly but is suffers from severe image artifacts. Recently, the deep learning based method for sparse-view CT reconstruction has attracted a major attention. However, neural networks often have a limited ability to remove the artifacts when they only work in the image domain. Deep learning-based sinogram processing can achieve a better anti-artifact performance, but it inevitably requires feature maps of the whole image in a video memory, which makes handling large-scale or three-dimensional (3D) images rather challeng…
$L_1$ regularization is used for finding sparse solutions to an underdetermined linear system. As sparse signals are widely expected in remote sensing, this type of regularization scheme and its extensions have been widely employed in many remote sensing problems, such as image fusion, target detection, image super-resolution, and others and have led to promising results. However, solving such sparse reconstruction problems is computationally expensive and has limitations in its practical use. In this paper, we proposed a novel efficient algorithm for solving the complex-valued $L_1$ regulariz…
A Conditional Denoising Diffusion Probabilistic Model for Radio Interferometric Image Reconstruction
The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；Hong Kong University of Science and Technology（香港科技大学）；Guangzhou University（广州大学）
Expert Validation of Synthetic Cervical Spine Radiographs Generated with a Denoising Diffusion Probabilistic Model
Cumming School of Medicine, University of Calgary（卡里尔大学医学学院）；Division of Neurosurgery, Department of Clinical Neurosciences, University of Calgary（卡里尔大学临床神经科学系神经外科部）；Department of Radiology, University of Calgary（卡里尔大学放射科）；Division of Neurosurgery, Department of Surgery, University of Toronto（多伦多大学外科系神经外科部）；Department of Medical Imaging, University of Toronto（多伦多大学医学影像科）；Project neuroArm, Department of Clinical Sciences, University of Calgary（卡里尔大学临床科学部neuroArm项目）
University of Alberta（阿尔伯塔大学）；v Geomatics（3v地理空间技术公司）
Interferometric Synthetic Aperture Radar (InSAR) imagery for estimating ground movement, based on microwaves reflected off ground targets is gaining increasing importance in remote sensing. However, noise corrupts microwave reflections received at satellite and contaminates the signal's wrapped phase. We introduce Convolutional Neural Networks (CNNs) to this problem domain and show the effectiveness of autoencoder CNN architectures to learn InSAR image denoising filters in the absence of clean ground truth images, and for artefact reduction in estimated coherence through intelligent preprocess…
