Diffusion models have attained prominence for their ability to synthesize a probability distribution for a given dataset via a diffusion process, enabling the generation of new data points with high fidelity. However, diffusion processes are prone to generating samples that reflect biases in a training dataset. To address this issue, we develop constrained diffusion models by imposing diffusion constraints based on desired distributions that are informed by requirements. Specifically, we cast the training of diffusion models under requirements as a constrained distribution optimization problem that aims to reduce the distribution difference between original and generated data while obeying constraints on the distribution of generated data. We show that our constrained diffusion models generate new data from a mixture data distribution that achieves the optimal trade-off among objective and constraints. To train constrained diffusion models, we develop a dual training algorithm and characterize the optimality of the trained constrained diffusion model. We empirically demonstrate the effectiveness of our constrained models in two constrained generation tasks: (i) we consider a dataset with one or more underrepresented classes where we train the model with constraints to ensure fairly sampling from all classes during inference; (ii) we fine-tune a pre-trained diffusion model to sample from a new dataset while avoiding overfitting.
Projected Coupled Diffusion for Test-Time Constrained Joint Generation
School of Computing, National University of Singapore(新加坡国立大学计算机学院);Faculty of Computer Science and Information Technology, Universiti Malaya(马来亚大学计算机科学与信息技术学院);Institute of Data Science, National University of Singapore(新加坡国立大学数据科学研究所)
Diffusion-based learning framework for Constrained Nonconvex Optimization with Weighted Bootstrapped Refinement
ShanghaiTech University(上海科技大学);MoE Key Laboratory of Intelligent Perception(智能感知MoE重点实验室);Human Machine Collaboration(人机协同);Shanghai Jiao Tong University(上海交通大学);China Mobile Communications Company Limited Research Institute(中国移动通信公司有限研究院);The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳))
ACPO: Anchor-Constrained Perceptual Optimization for Diffusion Models with No-Reference Quality Guidance
School of Electronic and Information Engineering, Anhui University(安徽大学电子与信息工程学院);Institute of Artificial Intelligence, Hefei Comprehensive National Science Center(合肥国家综合科学中心人工智能研究院);School of Cyber Science and Technology, University of Science and Technology of China(中国科学技术大学网络科学与技术学院)
DualDiT: A Conditional Dual-Output Diffusion Transformer for Joint OCT Image and Segmentation Mask Generation
Instituto Universitario de Investigación en Tecnología Centrada en el Ser Humano (HUMAN-tech), Universitat Politècnica de València(瓦伦西亚理工大学以人类为中心技术大学研究所);Artikode Intelligence S.L(Artikode智能公司);Biomedical Technology Center of the Medical Faculty, University of Muenster(明斯特大学医学院生物医学技术中心);Department of Ophthalmology, University of Muenster Medical Centre(明斯特大学医学中心眼科)
Score-based constrained generative modeling via Langevin diffusions with boundary conditions
Institute for Analysis and Scientific Computing, TU Wien(维也纳技术大学分析与科学计算研究所);Department of Mathematical Sciences, Chalmers University of Technology and the University of Gothenburg(查尔姆斯理工大学数学科学系)