arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2502.10330cs.LG
收藏

基于扩散的约束非凸优化学习框架与加权自举细化

Diffusion-based learning framework for Constrained Nonconvex Optimization with Weighted Bootstrapped Refinement

ShanghaiTech University(上海科技大学);MoE Key Laboratory of Intelligent Perception(智能感知MoE重点实验室);Human Machine Collaboration(人机协同);Shanghai Jiao Tong University(上海交通大学);China Mobile Communications Company Limited Research Institute(中国移动通信公司有限研究院);The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳))

Shutong Ding, Yimiao Zhou, Ke Hu, Xi Yao, Junchi Yan, Xiaoying Tang, Ye Shi

AI总结:

提出DiOpt框架,通过监督预热和自举训练两阶段学习噪声到约束区域的映射,解决扩散模型在约束非凸优化中的分布错位问题,实现高约束满足和最优性。

摘要:

扩散模型的最新进展显示出通过利用其多模态性来加速非凸问题求解的潜力。然而,现有的大多数基于扩散的优化方法依赖于监督学习,并且缺乏强制执行约束满足的机制,而这在现实应用中是需要满足的。在这种情况下,我们研究并理论分析了监督扩散求解器的固有问题,并识别出分布错位问题,即生成的解分布在可行区域上的概率质量通常较低。为了解决这个问题,我们提出了DiOpt,一种新的基于扩散的约束非凸优化学习框架,它有效地学习了从噪声到约束区域的映射。具体来说,该框架在两个不同的阶段运行:初始的预热阶段,通过监督学习实现,随后是自举训练阶段。这种双阶段架构旨在迭代地细化解,从而在高度满足约束的情况下改进目标函数。最后,我们还在推理中采用解选择技术以获得更好的最优性。值得注意的是,DiOpt是首次成功将扩散求解器集成到约束非凸优化中。在多样化的非凸任务上的评估显示了DiOpt在最优性和约束满足方面的优越性。我们的官方页面发布在https://dingsht.tech/diopt-webpage。

英文摘要:

Recent advances in diffusion models show promising potential to accelerate nonconvex problem solving by leveraging their multimodality. However, most existing diffusion-based optimization approaches rely on supervised learning and lack a mechanism to enforce constraint satisfaction, which is required in real-world applications. In that case, we investigate and theoretically analyze the inherent problem of supervised diffusion solvers and identify the distributional misalignment problem, i.e., the generated solution distribution often exhibits low probability mass on the feasible region. To resolve this issue, we propose DiOpt, a new diffusion-based learning framework for constrained nonconvex optimization, which effectively learns the mapping from noise to the constraint region. Specifically, this framework operates in two distinct phases: an initial warm-start phase, implemented via supervised learning, followed by a bootstrapping training phase. This dual-phase architecture is designed to iteratively refine solutions, thereby improving the objective function with high constraint satisfaction. Finally, we also employ a solution selection technique in inference for better optimality. Notably, DiOpt is the first successful integration of the diffusion solver in constrained nonconvex optimization. Evaluations on diverse nonconvex tasks demonstrate the superiority of DiOpt in both optimality and constraint satisfaction. Our official page is released at https://dingsht.tech/diopt-webpage.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑