Addressing real-world optimization problems becomes particularly challenging when analytic objective functions or constraints are unavailable. While numerous studies have addressed the issue of unknown objectives, limited research has focused on scenarios where feasibility constraints are not given explicitly. Overlooking these constraints can lead to spurious solutions that are unrealistic in practice. To deal with such unknown constraints, we propose to perform optimization within the data manifold using diffusion models. To constrain the optimization process to the data manifold, we reformulate the original optimization problem as a sampling problem from the product of the Boltzmann distribution defined by the objective function and the data distribution learned by the diffusion model. Depending on the differentiability of the objective function, we propose two different sampling methods. For differentiable objectives, we propose a two-stage framework that begins with a guided diffusion process for warm-up, followed by a Langevin dynamics stage for further correction. For non-differentiable objectives, we propose an iterative importance sampling strategy using the diffusion model as the proposal distribution. Comprehensive experiments on a synthetic dataset, six real-world black-box optimization datasets, and a multi-objective molecule optimization dataset show that our method achieves better or comparable performance with previous state-of-the-art baselines.
Score Mismatch and Set-Constrained Sampling in Zero-Shot Conditional Discrete Diffusion
University of California, Davis(加州大学戴维斯分校);Luddy School of Informatics, Computing, and Engineering, Indiana University Indianapolis(印第安纳大学印第安纳波利斯分校卢迪信息学、计算与工程学院);The Ohio State University(俄亥俄州立大学)
Constrained and Composite Sampling via Proximal Sampler
Department of Computer Science, University of Rochester(计算机科学系,罗切斯特大学);Goergen Institute for Data Science and Artificial Intelligence (GIDS-AI)(数据科学与人工智能研究所(GIDS-AI))
Diffusion Models Adapt to Low-Dimensional Structure Under Flexible Coefficient Choices
Department of Industrial and Operations Engineering, University of Michigan(工业与运营工程系,密歇根大学);Department of Statistics and Data Science, Chinese University of Hong Kong(统计与数据科学系,中国香港大学)
BFMT: Enhancing Search Capabilities of Tree Sampler via Bootstrap Flow-Map Tree
Department of CSE, Washington University in St.Louis, USA(美国圣路易斯华盛顿大学计算机科学与工程系);Department of Information Technology, Uppsala University, Sweden(瑞典乌普萨拉大学信息技术系)
Score-based constrained generative modeling via Langevin diffusions with boundary conditions
Institute for Analysis and Scientific Computing, TU Wien(维也纳技术大学分析与科学计算研究所);Department of Mathematical Sciences, Chalmers University of Technology and the University of Gothenburg(查尔姆斯理工大学数学科学系)
Trajectory Optimization for Unknown Constrained Systems using Reinforcement Learning
Information Technology R&D Center, Mitsubishi Electric Corporation(三菱电机公司信息技术研发中心);Mitsubishi Electric Research Labs(三菱电机研究实验室);Mitsubishi Electric Corp(三菱电机公司)
Locally Optimal Private Sampling: Beyond the Global Minimax
Department of Computing and Software(计算与软件系);McMaster University(麦斯特大学);Department of Mathematical Sciences(数学科学系);Korea Advanced Institute of Science & Technology(韩国科学技术院)