Polyp-DDPM: Diffusion-Based Semantic Polyp Synthesis for Enhanced Segmentation
National Taiwan University of Science and Technology（台湾科技大学）；National Taiwan University College of Medicine（台湾大学医学院）
Zolnamar Dorjsembe, Hsing-Kuo Pao, Furen Xiao
备注：
This preprint has been accepted for publication in the proceedings of the IEEE Engineering in Medicine and Biology Society (EMBC 2024). The final published version is available at https://doi.org/10.1109/EMBC53108.2024.10782077. The copyright for this work has been transferred to IEEE
发表信息：
Proceedings of the 46th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), 2024
This study introduces Polyp-DDPM, a diffusion-based method for generating realistic images of polyps conditioned on masks, aimed at enhancing the segmentation of gastrointestinal (GI) tract polyps. Our approach addresses the challenges of data limitations, high annotation costs, and privacy concerns associated with medical images. By conditioning the diffusion model on segmentation masks-binary masks that represent abnormal areas-Polyp-DDPM outperforms state-of-the-art methods in terms of image quality (achieving a Frechet Inception Distance (FID) score of 78.47, compared to scores above 83.79) and segmentation performance (achieving an Intersection over Union (IoU) of 0.7156, versus less than 0.6694 for synthetic images from baseline models and 0.7067 for real data). Our method generates a high-quality, diverse synthetic dataset for training, thereby enhancing polyp segmentation models to be comparable with real images and offering greater data augmentation capabilities to improve segmentation models. The source code and pretrained weights for Polyp-DDPM are made publicly available at https://github.com/mobaidoctor/polyp-ddpm.
Lesion-DDPM: Lesion-Enhanced 3D Diffusion for MS MRI Synthesis
University of Texas at Arlington（德克萨斯大学阿灵顿分校）；University of Texas at San Antonio（德克萨斯大学圣安东尼奥分校）；University of Texas at Dallas（德克萨斯大学达拉斯分校）；National Taiwan University Hospital（国立台湾大学医院）；National University of Mongolia（蒙古国立大学）；University of Texas at Austin（德克萨斯大学奥斯汀分校）
Cycle-guided Denoising Diffusion Probability Model for 3D Cross-modality MRI Synthesis
Emory University（埃默里大学）；Winship Cancer Institute（温希普癌症研究所）；Memorial Sloan Kettering Cancer Center（纪念斯隆凯特琳癌症中心）；Icahn School of Medicine at Mount Sinai（西奈山伊坎医学院）
Lung-DDPM+: Efficient Thoracic CT Image Synthesis using Diffusion Probabilistic Model
Centre de recherche du CHU de Québec-Université Laval（魁北克-拉瓦尔大学研究中心）；Department of Molecular Biology, Medical Biochemistry and Pathology, Université Laval（拉瓦尔大学分子生物学、医学生物化学与病理学系）；Cancer Research Center, Université Laval（拉瓦尔大学癌症研究中心）；Big Data Research Center, Université Laval（拉瓦尔大学大数据研究中心）；Department of Radiology, Carver College of Medicine - The University of Iowa（爱荷华大学卡弗医学院放射科）
Norwegian University of Science and Technology (NTNU)（挪威科技大学（NTNU））；Oslo University Hospital（奥斯陆大学医院）；University of Oslo (UiO)（奥斯陆大学（UiO））
One of the major obstacles in automatic polyp detection during colonoscopy is the lack of labeled polyp training images. In this paper, we propose a framework of conditional adversarial networks to increase the number of training samples by generating synthetic polyp images. Using a normal binary form of polyp mask which represents only the polyp position as an input conditioned image, realistic polyp image generation is a difficult task in a generative adversarial networks approach. We propose an edge filtering-based combined input conditioned image to train our proposed networks. This enable…
Lung-DDPM: Semantic Layout-guided Diffusion Models for Thoracic CT Image Synthesis
Centre de recherche du CHU de Québec-Université Laval（魁北克大学拉瓦尔医院研究中心）；Centre de recherche de l’Institut universitaire de cardiologie et de pneumologie de Québec-Université Laval（魁北克大学拉瓦尔心血管与呼吸病学研究所）
Kyungpook National University（庆北国立大学）；Electronics and Telecommunications Research Institute（韩国电子通信研究院）
Deep generative models for audio synthesis have recently been significantly improved. However, the task of modeling raw-waveforms remains a difficult problem, especially for audio waveforms and music signals. Recently, the realtime audio variational autoencoder (RAVE) method was developed for high-quality audio waveform synthesis. The RAVE method is based on the variational autoencoder and utilizes the two-stage training strategy. Unfortunately, the RAVE model is limited in reproducing wide-pitch polyphonic music sound. Therefore, to enhance the reconstruction performance, we adopt the pitch a…
CT Synthesis with Conditional Diffusion Models for Abdominal Lymph Node Segmentation
Shanghai Jiao Tong University, Shanghai, China（上海交通大学）；Department of Surgical Oncology and General Surgery, Key Laboratory of Precision Diagnosis and Treatment of Gastrointestinal Tumors, Ministry of Education, The First Hospital of China Medical University, Shenyang, China（外科肿瘤科和普通外科，精准诊断与治疗胃肠肿瘤国家重点实验室，教育部，中国医科大学第一附属医院，沈阳，中国）；Department of Radiology, The First Hospital of China Medical University, Shenyang, China（放射科，中国医科大学第一附属医院，沈阳，中国）；Shanghai AI Laboratory, Shanghai, China（上海人工智能实验室，上海，中国）
VS-DDPM: Efficient Low-Cost Diffusion Model for Medical Modality Translation
Center Algoritmi / LASI, University of Minho（中心Algoritmi / LASI，明霍大学）；Institute for Artificial Intelligence in Medicine (IKIM)（医学人工智能研究所）；Essen University Hospital (AöR)（埃森大学医院（AöR））；University of Duisburg-Essen（杜伊斯堡-埃森大学）；Cancer Research Center Cologne Essen (CCCE)（科隆-埃森癌症研究中心（CCCE））；West German Cancer Center（西德癌症中心）；University Hospital Essen (AöR)（埃森大学医院（AöR））；German Cancer Consortium (DKTK)（德国癌症联盟（DKTK））；Partner site University Hospital Essen (AöR)（埃森大学医院（AöR）合作站点）；Institute of Computer Graphics and Vision（计算机图形与视觉研究所）；Graz University of Technology（格拉茨技术大学）；Institute of Medical Informatics（医学信息学研究所）；University Hospital RWTH Aachen（亚琛RWTH大学医院）；Department of Oral and Maxillofacial Surgery（口腔和颌面外科部门）；TU Dortmund University（多特蒙德技术大学）；Center for Virtual and Extended Reality in Medicine (ZvRM)（医学虚拟和扩展现实中心（ZvRM））；Faculty of Computer Science（计算机科学学院）
Prompt to Polyp: Medical Text-Conditioned Image Synthesis with Diffusion Models
AIRI；HSE University（俄罗斯高等经济大学）；Simula Metropolitan Center for Digital Engineering（Simula 数字工程研究中心）；Oslo Metropolitan University（奥斯陆 Metropolitan 大学）；Kazan Federal University（卡扎ン联邦大学）
Retinal OCT Synthesis with Denoising Diffusion Probabilistic Models for Layer Segmentation
Institute of Imaging and Computer Vision, RWTH Aachen University（成像与计算机视觉研究所，亚琛RWTH大学）；Bio-manufacturing Engineering Laboratory of Tsinghua SIGS, Tsinghua University（清华大学SIGS生物制造工程实验室）；Shenzhen Eye Hospital（深圳眼科医院）；Department of Ophthalmology, RWTH Aachen University（眼科部，亚琛RWTH大学）
CCIS-Diff: A Generative Model with Stable Diffusion Prior for Controlled Colonoscopy Image Synthesis
School of Software Engineering, Xi'an Jiaotong University（西安交通大学软件工程学院）；Tsinghua Shenzhen International Graduate School, Tsinghua University（清华大学深圳国际研究生院）；Guangdong Laboratory of Artificial Intelligence and Digital Economy (SZ)（广东省人工智能与数字经济实验室（深圳））
Accelerating Redshift-Conditioned Galaxy Image Synthesis with One-step Generative Modeling
The Center for Computational Science（计算科学中心）；University College London（伦敦大学学院）；Cavendish Laboratory（卡文迪许实验室）；Kavli Institute for Cosmology University of Cambridge（剑桥大学卡文迪许宇宙研究所）
Denoising Diffusion Probabilistic Models (DDPMs) have emerged as a powerful family of generative models that can yield high-fidelity samples and competitive log-likelihoods across a range of domains, including image and speech synthesis. Key advantages of DDPMs include ease of training, in contrast to generative adversarial networks, and speed of generation, in contrast to autoregressive models. However, DDPMs typically require hundreds-to-thousands of steps to generate a high fidelity sample, making them prohibitively expensive for high dimensional problems. Fortunately, DDPMs allow trading g…
An ideal music synthesizer should be both interactive and expressive, generating high-fidelity audio in realtime for arbitrary combinations of instruments and notes. Recent neural synthesizers have exhibited a tradeoff between domain-specific models that offer detailed control of only specific instruments, or raw waveform models that can train on any music but with minimal control and slow generation. In this work, we focus on a middle ground of neural synthesizers that can generate audio from MIDI sequences with arbitrary combinations of instruments in realtime. This enables training on a wid…
Denoising diffusion probabilistic models (DDPMs) are a recent family of generative models that achieve state-of-the-art results. In order to obtain class-conditional generation, it was suggested to guide the diffusion process by gradients from a time-dependent classifier. While the idea is theoretically sound, deep learning-based classifiers are infamously susceptible to gradient-based adversarial attacks. Therefore, while traditional classifiers may achieve good accuracy scores, their gradients are possibly unreliable and might hinder the improvement of the generation results. Recent work dis…
Memorial Sloan Kettering Cancer Center（纪念斯隆凯特琳癌症中心）；Stony Brook University（石溪大学）
Colorectal cancer screening modalities, such as optical colonoscopy (OC) and virtual colonoscopy (VC), are critical for diagnosing and ultimately removing polyps (precursors of colon cancer). The non-invasive VC is normally used to inspect a 3D reconstructed colon (from CT scans) for polyps and if found, the OC procedure is performed to physically traverse the colon via endoscope and remove these polyps. In this paper, we present a deep learning framework, Extended and Directional CycleGAN, for lossy unpaired image-to-image translation between OC and VC to augment OC video sequences with scale…
Beihang University（北京航空航天大学）；Sun Yatsen University（中山大学）；University of Pisa（比萨大学）；University of Trento（特伦托大学）；Jiangxi Research Institute, Beihang University（北京航空航天大学江西研究院）
Diffusion-based generative models have recently emerged as powerful solutions for high-quality synthesis in multiple domains. Leveraging the bidirectional Markov chains, diffusion probabilistic models generate samples by inferring the reversed Markov chain based on the learned distribution mapping at the forward diffusion process. In this work, we propose Modiff, a conditional paradigm that benefits from the denoising diffusion probabilistic model (DDPM) to tackle the problem of realistic and diverse action-conditioned 3D skeleton-based motion generation. We are a pioneering attempt that uses …
Balancing Fidelity, Utility, and Privacy in Synthetic Cardiac MRI Generation: A Comparative Study
University of Peradeniya, Sri Lanka（斯里兰卡大学）；University of Maryland, College Park, USA（美国马里兰大学 College Park 分校）；Tulane University School of Medicine, USA（美国 Tulane 大学医学院）；Simula Research Laboratory, Norway（挪威 Simula 研究实验室）；SimulaMet, Norway（挪威 SimulaMet）
ArSDM: Colonoscopy Images Synthesis with Adaptive Refinement Semantic Diffusion Models
Shenzhen Research Institute of Big Data（深圳大数据研究院）；Shenzhen Health Development Research and Data Management Center（深圳市卫生健康发展研究和数据管理中心）；The Chinese University of Hong Kong（香港中文大学）；Sun Yat-sen University（中山大学）
University of California, Irvine（加州大学欧文分校）；Indian Institute of Technology, Kanpur（印度理工学院坎普尔分校）；Google Research（谷歌研究院）
Diffusion probabilistic models have been shown to generate state-of-the-art results on several competitive image synthesis benchmarks but lack a low-dimensional, interpretable latent space, and are slow at generation. On the other hand, standard Variational Autoencoders (VAEs) typically have access to a low-dimensional latent space but exhibit poor sample quality. We present DiffuseVAE, a novel generative framework that integrates VAE within a diffusion model framework, and leverage this to design novel conditional parameterizations for diffusion models. We show that the resulting model equips…
University of Science and Technology of China（中国科学技术大学）；Microsoft Research（微软研究院）；Microsoft Cloud+AI（微软云与人工智能部门）
We present the vector quantized diffusion (VQ-Diffusion) model for text-to-image generation. This method is based on a vector quantized variational autoencoder (VQ-VAE) whose latent space is modeled by a conditional variant of the recently developed Denoising Diffusion Probabilistic Model (DDPM). We find that this latent-space method is well-suited for text-to-image generation tasks because it not only eliminates the unidirectional bias with existing methods but also allows us to incorporate a mask-and-replace diffusion strategy to avoid the accumulation of errors, which is a serious problem w…
Diffusion models have recently exhibited remarkable abilities to synthesize striking image samples since the introduction of denoising diffusion probabilistic models (DDPMs). Their key idea is to disrupt images into noise through a fixed forward process and learn its reverse process to generate samples from noise in a denoising way. For conditional DDPMs, most existing practices relate conditions only to the reverse process and fit it to the reversal of unconditional forward process. We find this will limit the condition modeling and generation in a small time window. In this paper, we propose…
Oxford-Man Institute of Quantitative Finance, University of Oxford（牛津大学牛津-曼数量金融研究所）；Mathematical Institute, University of Oxford（牛津大学数学研究所）；University of Cambridge（剑桥大学）；Imperial College London（伦敦帝国学院）；Royal Brompton Hospital（皇家布朗普顿医院）；King’s College London（伦敦国王学院）
Zhejiang University（浙江大学）；Tencent AI Lab（腾讯人工智能实验室）
Denoising diffusion probabilistic models (DDPMs) have recently achieved leading performances in many generative tasks. However, the inherited iterative sampling process costs hindered their applications to speech synthesis. This paper proposes FastDiff, a fast conditional diffusion model for high-quality speech synthesis. FastDiff employs a stack of time-aware location-variable convolutions of diverse receptive field patterns to efficiently model long-term time dependencies with adaptive conditions. A noise schedule predictor is also adopted to reduce the sampling steps without sacrificing the…
DualDiT: A Conditional Dual-Output Diffusion Transformer for Joint OCT Image and Segmentation Mask Generation
Instituto Universitario de Investigación en Tecnología Centrada en el Ser Humano (HUMAN-tech), Universitat Politècnica de València（瓦伦西亚理工大学以人类为中心技术大学研究所）；Artikode Intelligence S.L（Artikode智能公司）；Biomedical Technology Center of the Medical Faculty, University of Muenster（明斯特大学医学院生物医学技术中心）；Department of Ophthalmology, University of Muenster Medical Centre（明斯特大学医学中心眼科）
Renmin University of China（中国人民大学）；University of Science and Technology of China（中国科学技术大学）；South China University of Technology（华南理工大学）；University of Surrey（萨里大学）；MSRA（微软亚洲研究院）；Microsoft Azure Speech（微软Azure语音）；Imperial College London（帝国理工学院）
Denoising Diffusion Probabilistic Models (DDPMs) are emerging in text-to-speech (TTS) synthesis because of their strong capability of generating high-fidelity samples. However, their iterative refinement process in high-dimensional data space results in slow inference speed, which restricts their application in real-time systems. Previous works have explored speeding up by minimizing the number of inference steps but at the cost of sample quality. In this work, to improve the inference speed for DDPM-based TTS model while achieving high sample quality, we propose ResGrad, a lightweight diffusi…
Adaptively Distilled ControlNet: Accelerated Training and Superior Sampling for Medical Image Synthesis
National University of Singapore（国立新加坡大学）；National University of Singapore Suzhou Research Institute（国立新加坡大学苏州市研究 institute）；City University of Hong Kong（香港城市大学）
fastWDM3D: Fast and Accurate 3D Healthy Tissue Inpainting
Department of Biomedical Engineering, University of Basel, Switzerland Department of Quantitative Biomedicine, University of Zurich, Switzerland Helmholtz AI, Helmholtz Zentrum München, Germany Department of Diagnostic；Interventional Neuroradiology, School of Medicine, Klinikum rechts der Isar, Technical University of Munich, Germany TranslaTUM - Central Institute for Translational Cancer Research, Technical University of Munich, Germany
