Diffusion-based image generation models can enhance image quality when conditioned on ground truth labels. Here, we conduct a comprehensive experimental study on image-level conditioning for diffusion models using cluster assignments. We investigate how individual clustering determinants, such as the number of clusters and the clustering method, impact image synthesis across three different datasets. Given the optimal number of clusters with respect to image synthesis, we show that cluster-conditioning can achieve state-of-the-art performance, with an FID of 1.67 for CIFAR10 and 2.17 for CIFAR100, along with a strong increase in training sample efficiency. We further propose a novel empirical method to estimate an upper bound for the optimal number of clusters. Unlike existing approaches, we find no significant association between clustering performance and the corresponding cluster-conditional FID scores. The code is available at https://github.com/HHU-MMBS/cedm-official-wavc2025.
Rethinking Real-world Image Deraining via An Unpaired Degradation-Conditioned Diffusion Model
Nanjing University of Aeronautics and Astronautics（南京航空航天大学）；The Hong Kong Polytechnic University（香港理工大学）；University of Science and Technology of China（中国科学技术大学）
ReImagine: Rethinking Controllable High-Quality Human Video Generation via Image-First Synthesis
School of Science and Engineering, The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳）科学与工程学院）；Future Network of Intelligence Institute, CUHK-Shenzhen, China（CUHK-深圳智能网络研究院）
National University of Singapore（新加坡国立大学）；Tencent PCG（腾讯平台与内容事业群）
Vector-Quantized (VQ-based) generative models usually consist of two basic components, i.e., VQ tokenizers and generative transformers. Prior research focuses on improving the reconstruction fidelity of VQ tokenizers but rarely examines how the improvement in reconstruction affects the generation ability of generative transformers. In this paper, we surprisingly find that improving the reconstruction fidelity of VQ tokenizers does not necessarily improve the generation. Instead, learning to compress semantic features within VQ tokenizers significantly improves generative transformers' ability …
Evidence-Based Text-Conditioned 3D CT Synthesis for Ovarian Cancer
AIRLab, Politecnico di Milano（米兰理工学院AIR实验室）；Politecnico di Milano（米兰理工学院）；NEARLab, Politecnico di Milano（米兰理工学院NEAR实验室）；Istituto Europeo di Oncologia（欧洲肿瘤研究所）；Università degli Studi dell’Insubria（意大利北部大学）；Karlsruhe Institute of Technology（卡尔斯鲁厄理工学院）
Rethinking Discrete Tokens: Treating Them as Conditions for Continuous Autoregressive Image Synthesis
School of Artificial Intelligence, Jilin University（吉林大学人工智能学院）；Shanghai Innovation Institute（上海创新研究院）；Institute of Trustworthy Embodied AI, Fudan University（复旦大学可信具身人工智能研究院）；Department of Computer Science, The University of Hong Kong（香港大学计算机科学系）
Exposure bias describes the phenomenon that a language model trained under the teacher forcing schema may perform poorly at the inference stage when its predictions are conditioned on its previous predictions unseen from the training corpus. Recently, several generative adversarial networks (GANs) and reinforcement learning (RL) methods have been introduced to alleviate this problem. Nonetheless, a common issue in RL and GANs training is the sparsity of reward signals. In this paper, we adopt two simple strategies, multi-range reinforcing, and multi-entropy sampling, to amplify and denoise the…
Deep learning is a popular and powerful tool in computed tomography (CT) image processing such as organ segmentation, but its requirement of large training datasets remains a challenge. Even though there is a large anatomical variability for children during their growth, the training datasets for pediatric CT scans are especially hard to obtain due to risks of radiation to children. In this paper, we propose a method to conditionally synthesize realistic pediatric CT images using a new auxiliary classifier generative adversarial network (ACGAN) architecture by taking age information into accou…
Offline reinforcement learning (Offline RL) suffers from the innate distributional shift as it cannot interact with the physical environment during training. To alleviate such limitation, state-based offline RL leverages a learned dynamics model from the logged experience and augments the predicted state transition to extend the data distribution. For exploiting such benefit also on the image-based RL, we firstly propose a generative model, S2P (State2Pixel), which synthesizes the raw pixel of the agent from its corresponding state. It enables bridging the gap between the state and the image d…
University College London（伦敦大学学院）；City University of Hong Kong（香港城市大学）；University College Hospital NHS foundation Trust（大学学院医院国民保健服务基金会信托）
We propose an image synthesis mechanism for multi-sequence prostate MR images conditioned on text, to control lesion presence and sequence, as well as to generate paired bi-parametric images conditioned on images e.g. for generating diffusion-weighted MR from T2-weighted MR for paired data, which are two challenging tasks in pathological image synthesis. Our proposed mechanism utilises and builds upon the recent stable diffusion model by proposing image-based conditioning for paired data generation. We validate our method using 2D image slices from real suspected prostate cancer patients. The …
Exploiting learning algorithms under scarce data regimes is a limitation and a reality of the medical imaging field. In an attempt to mitigate the problem, we propose a data augmentation protocol based on generative adversarial networks. We condition the networks at a pixel-level (segmentation mask) and at a global-level information (acquisition environment or lesion type). Such conditioning provides immediate access to the image-label pairs while controlling global class specific appearance of the synthesized images. To stimulate synthesis of the features relevant for the segmentation task, a…
Prompt to Polyp: Medical Text-Conditioned Image Synthesis with Diffusion Models
AIRI；HSE University（俄罗斯高等经济大学）；Simula Metropolitan Center for Digital Engineering（Simula 数字工程研究中心）；Oslo Metropolitan University（奥斯陆 Metropolitan 大学）；Kazan Federal University（卡扎ン联邦大学）
PCFlow: Physics-Conditioned Flow Matching for GPR B-Scan Image Synthesis
College of Computer Science and Technology, Jilin University（吉林大学计算机科学与技术学院）；Symbol Computation and Knowledge Engineering of the Ministry of Education, Jilin University（吉林大学符号计算与知识工程教育部重点实验室）
Rethinking Position Embedding as a Context Controller for Multi-Reference and Multi-Shot Video Generation
Wuhan University（武汉大学）；University of Science and Technology of China（中国科学技术大学）；Hong Kong University of Science and Technology（香港科技大学）；Tsinghua University（清华大学）
PHINN-EEG: Topological Time-Series Analysis of Dream-State EEG -- Dynamic Betti Curves for Dream Content Classification and Topology-Conditioned Neural Signal Synthesis
CloseUpShot: Close-up Novel View Synthesis from Sparse-views via Point-conditioned Diffusion Model
Shenzhen Future Network of Intelligence Institute (FNii-Shenzhen)（深圳未来网络智能研究院（FNii-深圳））；Chinese University of Hong Kong at Shenzhen (CUHKSZ)（香港中文大学（深圳））；Sun Yat-sen University（中山大学）
Accelerating Redshift-Conditioned Galaxy Image Synthesis with One-step Generative Modeling
The Center for Computational Science（计算科学中心）；University College London（伦敦大学学院）；Cavendish Laboratory（卡文迪许实验室）；Kavli Institute for Cosmology University of Cambridge（剑桥大学卡文迪许宇宙研究所）
University of Leeds（利兹大学）；KU Leuven（鲁汶大学）；UZ Leuven（鲁汶大学医院）
Generative models are undoubtedly a hot topic in Artificial Intelligence, among which the most common type is Generative Adversarial Networks (GANs). These architectures let one synthesise artificial datasets by implicitly modelling the underlying probability distribution of a real-world training dataset. With the introduction of Conditional GANs and their variants, these methods were extended to generating samples conditioned on ancillary information available for each sample within the dataset. From a practical standpoint, however, one might desire to generate data conditioned on partial inf…
Semantically Conditioned Diffusion Models for Cerebral DSA Synthesis
Department of Statistics, LMU Munich（统计学系）；Department of Biomedical Informatics (DBMI), Medical Faculty Mannheim, Heidelberg University（生物医学信息学系）；Munich Center for Machine Learning (MCML)（慕尼黑机器学习中心）；Clinic for Diagnostic and Interventional Neuroradiology, Medical Faculty Mannheim, Heidelberg University（诊断和介入神经放射科）；Department of Anesthesiology and Intensive Care Medicine, BG Trauma Center Tuebingen（麻醉和重症医学科）
Not All Actions Are Equal: Rethinking Conditioning for Dexterous World Model
The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；Shenzhen University（深圳大学）；Beijing University of Technology（北京工业大学）；JD Explore Academy（京东探索研究院）
Meta AI；LTCI, Télécom Paris, Institut Polytechnique de Paris（巴黎理工学院巴黎电信学院LTCI）
Generative image models have been extensively studied in recent years. In the unconditional setting, they model the marginal distribution from unlabelled images. To allow for more control, image synthesis can be conditioned on semantic segmentation maps that instruct the generator the position of objects in the image. While these two tasks are intimately related, they are generally studied in isolation. We propose OCO-GAN, for Optionally COnditioned GAN, which addresses both tasks in a unified manner, with a shared image synthesis network that can be conditioned either on semantic maps or dire…
Generative Adversarial Networks (GANs) have proven successful for unsupervised image generation. Several works have extended GANs to image inpainting by conditioning the generation with parts of the image to be reconstructed. Despite their success, these methods have limitations in settings where only a small subset of the image pixels is known beforehand. In this paper we investigate the effectiveness of conditioning GANs when very few pixel values are provided. We propose a modelling framework which results in adding an explicit cost term to the GAN objective function to enforce pixel-wise c…
Electrocardiogram (ECG) synthesis is the area of research focused on generating realistic synthetic ECG signals for medical use without concerns over annotation costs or clinical data privacy restrictions. Traditional ECG generation models consider a single ECG lead and utilize GAN-based generative models. These models can only generate single lead samples and require separate training for each diagnosis class. The diagnosis classes of ECGs are insufficient to capture the intricate differences between ECGs depending on various features (e.g. patient demographic details, co-existing diagnosis c…
Massachusetts Institute of Technology（麻省理工学院）；Siemens Technology（西门子技术公司）
Typical engineering design tasks require the effort to modify designs iteratively until they meet certain constraints, i.e., performance or attribute requirements. Past work has proposed ways to solve the inverse design problem, where desired designs are directly generated from specified requirements, thus avoid the trial and error process. Among those approaches, the conditional deep generative model shows great potential since 1) it works for complex high-dimensional designs and 2) it can generate multiple alternative designs given any condition. In this work, we propose a conditional deep g…
T-CACE: A Time-Conditioned Autoregressive Contrast Enhancement Multi-Task Framework for Contrast-Free Liver MRI Synthesis, Segmentation, and Diagnosis
Department of Computer Science, Toronto Metropolitan University（计算机科学系，多伦多 Metropolitan 大学）；School of Biomedical Engineering, Western University（生物医学工程学院，西部大学）
We present Poly-GAN, a novel conditional GAN architecture that is motivated by Fashion Synthesis, an application where garments are automatically placed on images of human models at an arbitrary pose. Poly-GAN allows conditioning on multiple inputs and is suitable for many tasks, including image alignment, image stitching, and inpainting. Existing methods have a similar pipeline where three different networks are used to first align garments with the human pose, then perform stitching of the aligned garment and finally refine the results. Poly-GAN is the first instance where a common architect…
