Medical imaging applications are highly specialized in terms of human anatomy, pathology, and imaging domains. Therefore, annotated training datasets for training deep learning applications in medical imaging not only need to be highly accurate but also diverse and large enough to encompass almost all plausible examples with respect to those specifications. We argue that achieving this goal can be facilitated through a controlled generation framework for synthetic images with annotations, requiring multiple conditional specifications as input to provide control. We employ a Denoising Diffusion Probabilistic Model (DDPM) to train a large-scale generative model in the lung CT domain and expand upon a classifier-free sampling strategy to showcase one such generation framework. We show that our approach can produce annotated lung CT images that can faithfully represent anatomy, convincingly fooling experts into perceiving them as real. Our experiments demonstrate that controlled generative frameworks of this nature can surpass nearly every state-of-the-art image generative model in achieving anatomical consistency in generated medical images when trained on comparable large medical datasets.
Beihang University（北京航空航天大学）；Sun Yatsen University（中山大学）；University of Pisa（比萨大学）；University of Trento（特伦托大学）；Jiangxi Research Institute, Beihang University（北京航空航天大学江西研究院）
Diffusion-based generative models have recently emerged as powerful solutions for high-quality synthesis in multiple domains. Leveraging the bidirectional Markov chains, diffusion probabilistic models generate samples by inferring the reversed Markov chain based on the learned distribution mapping at the forward diffusion process. In this work, we propose Modiff, a conditional paradigm that benefits from the denoising diffusion probabilistic model (DDPM) to tackle the problem of realistic and diverse action-conditioned 3D skeleton-based motion generation. We are a pioneering attempt that uses …
Peking University（北京大学）；Massachusetts General Hospital（麻省总医院）；Harvard Medical School（哈佛医学院）
In supervised learning for image denoising, usually the paired clean images and noisy images are collected or synthesised to train a denoising model. L2 norm loss or other distance functions are used as the objective function for training. It often leads to an over-smooth result with less image details. In this paper, we regard the denoising task as a problem of estimating the posterior distribution of clean images conditioned on noisy images. We apply the idea of diffusion model to realize generative image denoising. According to the noise model in denoising tasks, we redefine the diffusion p…
We propose a novel unpaired image-to-image translation method that uses denoising diffusion probabilistic models without requiring adversarial training. Our method, UNpaired Image Translation with Denoising Diffusion Probabilistic Models (UNIT-DDPM), trains a generative model to infer the joint distribution of images over both domains as a Markov chain by minimising a denoising score matching objective conditioned on the other domain. In particular, we update both domain translation models simultaneously, and we generate target domain images by a denoising Markov Chain Monte Carlo approach tha…
Multimodal-Conditioned Latent Diffusion Models for Fashion Image Editing
Media Integration and Communication Center (MICC), University of Florence（佛罗伦萨大学媒体集成与传播中心（MICC））；University of Florence（佛罗伦萨大学）；University of Modena and Reggio Emilia（摩德纳-雷焦艾米利亚大学）；University of Pisa（比萨大学）
We introduce a new method to efficiently create text-to-image models from a pre-trained CLIP and StyleGAN. It enables text driven sampling with an existing generative model without any external data or fine-tuning. This is achieved by training a diffusion model conditioned on CLIP embeddings to sample latent vectors of a pre-trained StyleGAN, which we call clip2latent. We leverage the alignment between CLIP's image and text embeddings to avoid the need for any text labelled data for training the conditional diffusion model. We demonstrate that clip2latent allows us to generate high-resolution …
AnF-DiffPET: Anatomy- and Frequency-Guided Diffusion for PET/CT Denoising
College of Medicine and Biological Information Engineering, Northeastern University（东北大学医学与生物信息工程学院）；Tohoku University School of Medicine（东北大学医学院）；Department of Clinical Neurosciences, University of Cambridge（剑桥大学临床神经科学系）
T2LDM++: A Self-Conditioned Representation Guided Diffusion Model for Realistic Text-to-LiDAR Scene Generation
School of Computer Science and Engineering（计算机科学与工程学院）；Nanjing University of Science and Technology（南京理工大学）；Faculty of Data Science（数据科学学院）；City University of Macau（澳门城市大学）；Beijing Institute of Technology（北京理工大学）；Fondazione Bruno Kessler（布鲁诺·凯瑟尔基金会）；School of Public Health（公共卫生学院）；Shanghai Jiao Tong University（上海交通大学）；School of Computing（计算机学院）；National University of Singapore（新加坡国立大学）
Speech Driven Video Editing via an Audio-Conditioned Diffusion Model
University of Galway（高威大学）；University of Wrocław（弗罗茨瓦夫大学）；Wrocław University of Science and Technology（弗罗茨瓦夫理工大学）；Tooploox（Tooploox公司）；Trinity College Dublin（都柏林圣三一学院）
The paper presents a novel approach for vector-floorplan generation via a diffusion model, which denoises 2D coordinates of room/door corners with two inference objectives: 1) a single-step noise as the continuous quantity to precisely invert the continuous forward process; and 2) the final 2D coordinate as the discrete quantity to establish geometric incident relationships such as parallelism, orthogonality, and corner-sharing. Our task is graph-conditioned floorplan generation, a common workflow in floorplan design. We represent a floorplan as 1D polygonal loops, each of which corresponds to…
Conditional generative models of high-dimensional images have many applications, but supervision signals from conditions to images can be expensive to acquire. This paper describes Diffusion-Decoding models with Contrastive representations (D2C), a paradigm for training unconditional variational autoencoders (VAEs) for few-shot conditional image generation. D2C uses a learned diffusion-based prior over the latent representations to improve generation and contrastive self-supervised learning to improve representation quality. D2C can adapt to novel generation tasks conditioned on labels or mani…
MDAA-Diff: CT-Guided Multi-Dose Adaptive Attention Diffusion Model for PET Denoising
School of Biomedical Engineering, Southern Medical University（南方医科大学生物医学工程学院）；School of Biomedical Engineering, Shanghai Jiao Tong University（上海交通大学生物医学工程学院）；Nanfang PET Center, Nanfang Hospital Southern Medical University（南方医科大学南芳医院PET中心）
3D Wavelet-Based Structural Priors for Controlled Diffusion in Whole-Body Low-Dose PET Denoising
School of Engineering, Zurich University of Applied Sciences, CH；Bioengineering Department；Imperial-X, Imperial College London, UK；DAMTP, University of Cambridge, UK；Lucerne University Teaching；Research Hospital, CH；Lung Institute, Imperial College London, UK；Cardiovascular Research Centre, Royal Brompton Hospital, UK；School of Biomedical Engineering \& Imaging Sciences, King's College London, UK；Yau Mathematical Sciences Center, Tsinghua University, CN
