Denoising Diffusion Models for Anomaly Localization in Medical Images
School of Computation, Information and Technology, Technical University of Munich（慕尼黑技术大学计算、信息与技术学院）；Institute of Machine Learning in Biomedical Imaging, Helmholtz Munich, Germany（生物医学影像机器学习研究所，海德堡慕尼黑德国）；Department of Biomedical Engineering, University of Basel, Allschwil, Switzerland（巴塞尔大学生物医学工程系，瑞士Allschwil）；School of Biomedical Engineering and Imaging Sciences, King’s College London, United Kingdom（伦敦国王学院生物医学工程与影像科学学院，英国）；Department of Biomedical Informatics and Data Science, Yale University School of Medicine, New Haven, CT, USA（耶鲁大学医学院生物医学信息学与数据科学系，美国新罕布什尔州新 Haven）；Yale Biomedical Imaging Institute, Yale University, New Haven, CT, USA（耶鲁大学生物医学影像研究院，美国新罕布什尔州新 Haven）
Cosmin I. Bercea, Philippe C. Cattin, Julia A. Schnabel, Julia Wolleb
This review explores anomaly localization in medical images using denoising diffusion models. After providing a brief methodological background of these models, including their application to image reconstruction and their conditioning using guidance mechanisms, we provide an overview of available datasets and evaluation metrics suitable for their application to anomaly localization in medical images. In this context, we discuss supervision schemes ranging from fully supervised segmentation to semi-supervised, weakly supervised, self-supervised, and unsupervised methods, and provide insights into the effectiveness and limitations of these approaches. Furthermore, we highlight open challenges in anomaly localization, including detection bias, domain shift, computational cost, and model interpretability. Our goal is to provide an overview of the current state of the art in the field, outline research gaps, and highlight the potential of diffusion models for robust anomaly localization in medical images.
In medical applications, weakly supervised anomaly detection methods are of great interest, as only image-level annotations are required for training. Current anomaly detection methods mainly rely on generative adversarial networks or autoencoder models. Those models are often complicated to train or have difficulties to preserve fine details in the image. We present a novel weakly supervised anomaly detection method based on denoising diffusion implicit models. We combine the deterministic iterative noising and denoising scheme with classifier guidance for image-to-image translation between d…
Spatial-aware Attention Generative Adversarial Network for Semi-supervised Anomaly Detection in Medical Image
School of Computer Science, Wuhan University（武汉大学计算机学院）；School of Computer Science, Central China Normal University（华中师范大学计算机学院）；University of Louisville（路易斯维尔大学）
Zhejiang University-University of Illinois Urbana-Champaign Institute（浙江大学-伊利诺伊大学厄巴纳-香槟分校联合学院）；Zhejiang University（浙江大学）
Reducing the radiation dose in computed tomography (CT) is important to mitigate radiation-induced risks. One option is to employ a well-trained model to compensate for incomplete information and map sparse-view measurements to the CT reconstruction. However, reconstruction from sparsely sampled measurements is insufficient to uniquely characterize an object in CT, and a learned prior model may be inadequate for unencountered cases. Medical modal translation from magnetic resonance imaging (MRI) to CT is an alternative but may introduce incorrect information into the synthesized CT images in a…
Pancreatic Tumor Segmentation as Anomaly Detection in CT Images Using Denoising Diffusion Models
School of Electrical and Computer Engineering, University of Oklahoma（俄克拉荷马大学电气与计算机工程学院）；University of Oklahoma Health Sciences Center（俄克拉荷马大学健康科学中心）；School of Computer Science, University of Oklahoma（俄克拉荷马大学计算机科学学院）
Magnetic resonance imaging (MRI) is a common and life-saving medical imaging technique. However, acquiring high signal-to-noise ratio MRI scans requires long scan times, resulting in increased costs and patient discomfort, and decreased throughput. Thus, there is great interest in denoising MRI scans, especially for the subtype of diffusion MRI scans that are severely SNR-limited. While most prior MRI denoising methods are supervised in nature, acquiring supervised training datasets for the multitude of anatomies, MRI scanners, and scan parameters proves impractical. Here, we propose Denoising…
Ambient Denoising Diffusion Generative Adversarial Networks for Establishing Stochastic Object Models from Noisy Image Data
Global Institute of Future Technology, Shanghai Jiao Tong University（上海交通大学未来技术研究院）；University of Michigan-Shanghai Jiao Tong University Joint Institute, Shanghai Jiao Tong University（上海交通大学密歇根大学联合学院）；University of Arizona（亚利桑那大学）；Wyant College of Optical Sciences, University of Arizona（亚利桑那大学怀恩特光学科学学院）
LPIXEL Inc.（LPIXEL公司）；University of Cambridge（剑桥大学）；National Institute of Informatics（国立信息学研究所）；National Center for Global Health and Medicine（国立国际医疗研究中心）；Juntendo University（顺天堂大学）；ELTE Eötvös Loránd University（厄特沃什·罗兰大学）；The University of Tokyo（东京大学）
Unsupervised learning can discover various unseen abnormalities, relying on large-scale unannotated medical images of healthy subjects. Towards this, unsupervised methods reconstruct a 2D/3D single medical image to detect outliers either in the learned feature space or from high reconstruction loss. However, without considering continuity between multiple adjacent slices, they cannot directly discriminate diseases composed of the accumulation of subtle anatomical anomalies, such as Alzheimer's Disease (AD). Moreover, no study has shown how unsupervised anomaly detection is associated with eith…
Harbin Institute of Technology at Shenzhen（哈尔滨工业大学（深圳））；Peng Cheng Laboratory（鹏城实验室）
Based on the Denoising Diffusion Probabilistic Model (DDPM), medical image segmentation can be described as a conditional image generation task, which allows to compute pixel-wise uncertainty maps of the segmentation and allows an implicit ensemble of segmentations to boost the segmentation performance. However, DDPM requires many iterative denoising steps to generate segmentations from Gaussian noise, resulting in extremely inefficient inference. To mitigate the issue, we propose a principled acceleration strategy, called pre-segmentation diffusion sampling DDPM (PD-DDPM), which is specially …
Birla Institute of Technology Science Pilani（比尔拉理工学院皮拉尼分校）
In this work, we present Eformer - Edge enhancement based transformer, a novel architecture that builds an encoder-decoder network using transformer blocks for medical image denoising. Non-overlapping window-based self-attention is used in the transformer block that reduces computational requirements. This work further incorporates learnable Sobel-Feldman operators to enhance edges in the image and propose an effective way to concatenate them in the intermediate layers of our architecture. The experimental analysis is conducted by comparing deterministic learning and residual learning for the …
Peking University（北京大学）；Academy for Advanced Interdisciplinary Studies（前沿交叉学科研究院）；Massachusetts General Hospital（麻省总医院）；Harvard Medical School（哈佛医学院）
We propose a novel and unified method, measurement-conditioned denoising diffusion probabilistic model (MC-DDPM), for under-sampled medical image reconstruction based on DDPM. Different from previous works, MC-DDPM is defined in measurement domain (e.g. k-space in MRI reconstruction) and conditioned on under-sampling mask. We apply this method to accelerate MRI reconstruction and the experimental results show excellent performance, outperforming full supervision baseline and the state-of-the-art score-based reconstruction method. Due to its generative nature, MC-DDPM can also quantify the unce…
Uncertainty quantification in inverse medical imaging tasks with deep learning has received little attention. However, deep models trained on large data sets tend to hallucinate and create artifacts in the reconstructed output that are not anatomically present. We use a randomly initialized convolutional network as parameterization of the reconstructed image and perform gradient descent to match the observation, which is known as deep image prior. In this case, the reconstruction does not suffer from hallucinations as no prior training is performed. We extend this to a Bayesian approach with M…
Image denoising is of great importance for medical imaging system, since it can improve image quality for disease diagnosis and downstream image analyses. In a variety of applications, dynamic imaging techniques are utilized to capture the time-varying features of the subject, where multiple images are acquired for the same subject at different time points. Although signal-to-noise ratio of each time frame is usually limited by the short acquisition time, the correlation among different time frames can be exploited to improve denoising results with shared information across time frames. With t…
Canon Medical Research Europe（佳能医疗欧洲研究中心）；University of Glasgow（格拉斯哥大学）；University of Edinburgh（爱丁堡大学）；The Alan Turing Institute（艾伦·图灵研究所）
Pathological brain lesions exhibit diverse appearance in brain images, in terms of intensity, texture, shape, size, and location. Comprehensive sets of data and annotations are difficult to acquire. Therefore, unsupervised anomaly detection approaches have been proposed using only normal data for training, with the aim of detecting outlier anomalous voxels at test time. Denoising methods, for instance classical denoising autoencoders (DAEs) and more recently emerging diffusion models, are a promising approach, however naive application of pixelwise noise leads to poor anomaly detection perform…
Task-based Regularization in Penalized Least-Squares for Binary Signal Detection Tasks in Medical Image Denoising
University of Michigan-Shanghai Jiao Tong University Joint Institute（密歇根大学-上海交通大学联合研究院）；Global Institute of Future Technology, Shanghai Jiao Tong University（上海交通大学未来科技全球研究院）；Wyant College of Optical Sciences, University of Arizona（亚利桑那大学怀恩特光学科学学院）；Department of Medical Imaging, University of Arizona（亚利桑那大学医学影像系）
Hong Kong Baptist University（香港浸会大学）；Aerospace Information Research Institute, Chinese Academy of Sciences（中国科学院空天信息创新研究院）；University of Chinese Academy of Sciences（中国科学院大学）；Shenzhen University（深圳大学）；Instituto de Telecomunicações（电信研究院）；Instituto Superior Técnico（里斯本高等技术学院）；Universidade de Lisboa（里斯本大学）
Hyperspectral imaging measures the amount of electromagnetic energy across the instantaneous field of view at a very high resolution in hundreds or thousands of spectral channels. This enables objects to be detected and the identification of materials that have subtle differences between them. However, the increase in spectral resolution often means that there is a decrease in the number of photons received in each channel, which means that the noise linked to the image formation process is greater. This degradation limits the quality of the extracted information and its potential applications…
Southern University of Science and Technology（南方科技大学）；University of British Columbia（不列颠哥伦比亚大学）
Due to the high cost of manually annotating medical images, especially for large-scale datasets, anomaly detection has been explored through training models with only normal data. Lacking prior knowledge of true anomalies is the main reason for the limited application of previous anomaly detection methods, especially in the medical image analysis realm. In this work, we propose a one-shot anomaly detection framework, namely LesionPaste, that utilizes true anomalies from a single annotated sample and synthesizes artificial anomalous samples for anomaly detection. First, a lesion bank is constru…
We cast the problem of image denoising as a domain translation problem between high and low noise domains. By modifying the cycleGAN model, we are able to learn a mapping between these domains on unpaired retinal optical coherence tomography images. In quantitative measurements and a qualitative evaluation by ophthalmologists, we show how this approach outperforms other established methods. The results indicate that the network differentiates subtle changes in the level of noise in the image. Further investigation of the model's feature maps reveals that it has learned to distinguish retinal l…
UiT The Arctic University of Norway（特罗姆瑟大学（挪威北极圈大学））
Recent work has shown that label-efficient few-shot learning through self-supervision can achieve promising medical image segmentation results. However, few-shot segmentation models typically rely on prototype representations of the semantic classes, resulting in a loss of local information that can degrade performance. This is particularly problematic for the typically large and highly heterogeneous background class in medical image segmentation problems. Previous works have attempted to address this issue by learning additional prototypes for each class, but since the prototypes are based on…
National Tsing Hua University（国立清华大学）；Florida International University（佛罗里达国际大学）；Lehigh University（理海大学）；Guangdong Provincial People’s Hospital（广东省人民医院）；Guangdong Academy of Medical Science（广东省医学科学院）；University of Notre Dame（圣母大学）
Deep learning had already demonstrated its power in medical images, including denoising, classification, segmentation, etc. All these applications are proposed to automatically analyze medical images beforehand, which brings more information to radiologists during clinical assessment for accuracy improvement. Recently, many medical denoising methods had shown their significant artifact reduction result and noise removal both quantitatively and qualitatively. However, those existing methods are developed around human-vision, i.e., they are designed to minimize the noise effect that can be perce…
Emory University（埃默里大学）；Indiana University-Purdue University Indianapolis（印第安纳大学-普渡大学印第安纳波利斯分校）；Arizona State University（亚利桑那州立大学）
Detecting out-of-distribution (OOD) samples in medical imaging plays an important role for downstream medical diagnosis. However, existing OOD detectors are demonstrated on natural images composed of inter-classes and have difficulty generalizing to medical images. The key issue is the granularity of OOD data in the medical domain, where intra-class OOD samples are predominant. We focus on the generalizability of OOD detection for medical images and propose a self-supervised Cascade Variational autoencoder-based Anomaly Detector (CVAD). We use a variational autoencoders' cascade architecture, …
Deep convolutional neural networks (DCNNs) have contributed many breakthroughs in segmentation tasks, especially in the field of medical imaging. However, \textit{domain shift} and \textit{corrupted annotations}, which are two common problems in medical imaging, dramatically degrade the performance of DCNNs in practice. In this paper, we propose a novel robust cross-denoising framework using two peer networks to address domain shift and corrupted label problems with a peer-review strategy. Specifically, each network performs as a mentor, mutually supervised to learn from reliable samples selec…
Center for Advanced Medical Computing and Analysis, Massachusetts General Hospital and Harvard Medical School（麻省总医院与哈佛医学院高级医学计算与分析中心）；Gordon Center for Medical Imaging, Massachusetts General Hospital and Harvard Medical School（麻省总医院与哈佛医学院戈登医学成像中心）
Deep neural networks have been proved efficient for medical image denoising. Current training methods require both noisy and clean images. However, clean images cannot be acquired for many practical medical applications due to naturally noisy signal, such as dynamic imaging, spectral computed tomography, arterial spin labeling magnetic resonance imaging, etc. In this paper we proposed a training method which learned denoising neural networks from noisy training samples only. Training data in the acquisition domain was split to two subsets and the network was trained to map one noisy set to the…
University Hospital Aachen（亚琛大学医院）；RWTH Aachen University（亚琛工业大学）；Ocumeda AG（奥库梅达股份公司）；University Hospital Heidelberg（海德堡大学医院）；University Hospital Würzburg（维尔茨堡大学医院）；University of Mainz（美因茨大学）；RWTH Aachen（亚琛工业大学）
Recent advances in computer vision have shown promising results in image generation. Diffusion probabilistic models in particular have generated realistic images from textual input, as demonstrated by DALL-E 2, Imagen and Stable Diffusion. However, their use in medicine, where image data typically comprises three-dimensional volumes, has not been systematically evaluated. Synthetic images may play a crucial role in privacy preserving artificial intelligence and can also be used to augment small datasets. Here we show that diffusion probabilistic models can synthesize high quality medical imagi…
RelativeFlow: Taming Medical Image Denoising Learning with Noisy Reference
School of Computer Science and Engineering, Southeast University, China（东南大学计算机科学与工程学院，中国）；Department of Biomedical Engineering, Case Western Reserve University, USA（凯斯西储大学生物医学工程系，美国）；School of Instrument Science and Engineering, Southeast University, China（东南大学仪器科学与工程学院，中国）；Department of Optical Sciences, University of Arizona, USA（亚利桑那大学光学科学系，美国）
Synomaly Noise and Multi-Stage Diffusion: A Novel Approach for Unsupervised Anomaly Detection in Medical Images
Computer Aided Medical Procedures, Technical University of Munich, Munich, Germany（技术大学慕尼黑计算机辅助医疗程序）；Munich Center of Machine Learning, Munich, Germany（慕尼黑机器学习中心）；Department for Vascular and Endovascular Surgery, rechts der Isar University Hospital, Technical University of Munich, Munich, Germany（技术大学慕尼黑血管和内血管外科部门）
The extensive use of medical CT has raised a public concern over the radiation dose to the patient. Reducing the radiation dose leads to increased CT image noise and artifacts, which can adversely affect not only the radiologists judgement but also the performance of downstream medical image analysis tasks. Various low-dose CT denoising methods, especially the recent deep learning based approaches, have produced impressive results. However, the existing denoising methods are all downstream-task-agnostic and neglect the diverse needs of the downstream applications. In this paper, we introduce a…
A Unified Score Matching Paradigm for Video Anomaly Detection and Anticipation
National Engineering Laboratory for Integrated Aero-Space-Ground-Ocean Big Data Application Technology（空天地海一体化大数据应用技术国家工程实验室）；School of Computer Science, Northwestern Polytechnical University（西北工业大学计算机学院）；School of Electronics and Information, Northwestern Polytechnical University（西北工业大学电子信息学院）
