arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2402.18012cs.LGcs.AI
收藏

扩散模型作为未知约束优化问题的约束采样器

Diffusion Models as Constrained Samplers for Optimization with Unknown Constraints

Georgia Tech(佐治亚理工学院);Cornell University(康奈尔大学);Université de Montréal(蒙特利尔大学);Mila(Mila研究院);ENS, CNRS, PSL University(巴黎高等师范学院、国家科学研究中心、巴黎-萨克勒大学);UCSD(加州大学圣地亚哥分校)

Lingkai Kong, Yuanqi Du, Wenhao Mu, Kirill Neklyudov, Valentin De Bortoli, Dongxia Wu, Haorui Wang, Aaron Ferber, Yi-An Ma, Carla P. Gomes, Chao Zhang

AI总结:

针对可行性约束未知的优化问题,本文利用扩散模型将优化重构为 Boltzmann 分布与数据分布乘积上的采样,并按目标可微性分别提出两阶段引导扩散加 Langevin dynamics 方法及迭代重要性采样方法,在多类基准上达到更优或相当性能。

摘要:

当解析形式的目标函数或约束不可用时,求解现实世界优化问题会变得尤其具有挑战性。尽管已有大量研究处理目标函数未知的问题,但针对可行性约束未被显式给出的情形,相关研究仍然有限。忽视这些约束可能产生在实践中并不现实的伪解。为处理此类未知约束,我们提出利用扩散模型在数据流形内执行优化。为了将优化过程约束在数据流形上,我们将原优化问题重新表述为一个采样问题,即从由目标函数定义的 Boltzmann 分布与扩散模型学习到的数据分布的乘积分布中采样。根据目标函数的可微性,我们提出两种不同的采样方法。对于可微目标,我们提出一个两阶段框架:首先通过引导扩散过程进行预热,随后进入 Langevin dynamics 阶段作进一步修正。对于不可微目标,我们提出一种以扩散模型作为提议分布的迭代重要性采样策略。在一个合成数据集、六个现实世界黑盒优化数据集以及一个多目标分子优化数据集上的综合实验表明,与以往最先进基线相比,我们的方法取得了更优或相当的性能。

英文摘要:

Addressing real-world optimization problems becomes particularly challenging when analytic objective functions or constraints are unavailable. While numerous studies have addressed the issue of unknown objectives, limited research has focused on scenarios where feasibility constraints are not given explicitly. Overlooking these constraints can lead to spurious solutions that are unrealistic in practice. To deal with such unknown constraints, we propose to perform optimization within the data manifold using diffusion models. To constrain the optimization process to the data manifold, we reformulate the original optimization problem as a sampling problem from the product of the Boltzmann distribution defined by the objective function and the data distribution learned by the diffusion model. Depending on the differentiability of the objective function, we propose two different sampling methods. For differentiable objectives, we propose a two-stage framework that begins with a guided diffusion process for warm-up, followed by a Langevin dynamics stage for further correction. For non-differentiable objectives, we propose an iterative importance sampling strategy using the diffusion model as the proposal distribution. Comprehensive experiments on a synthetic dataset, six real-world black-box optimization datasets, and a multi-objective molecule optimization dataset show that our method achieves better or comparable performance with previous state-of-the-art baselines.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑