基于共识的优化方法全局收敛
Consensus-Based Optimization Methods Converge Globally
AI总结:
本文提出平均场分析技术,证明多智能体无导数CBO在智能体数趋于无穷时对非凸、非光滑及局部Lipschitz目标全局收敛,并给出非渐近Laplace原理与概率估计。
摘要:
本文研究基于共识的优化(CBO),这是一种多智能体元启发式无导数优化方法,能够对非凸非光滑函数进行全局最小化,并且适合理论分析。基于一个有实验支持的直觉,即平均而言CBO执行的是到全局极小点的平方欧氏距离的梯度下降,我们设计了一种新技术,用于证明在一大类目标函数下平均场定律收敛到全局极小点。该结果揭示了促成CBO方法成功的内部机制。特别地,我们证明了当优化智能体数量趋于无穷大时,CBO会对一大类优化问题执行凸化。此外,我们改进了以往的分析:对方法的初始化只要求温和假设,并且覆盖仅局部Lipschitz连续的目标函数。作为该分析的核心组成部分,我们建立了一个定量的非渐近Laplace原理,这一原理本身可能具有独立意义。由CBO在平均场定律下的收敛结果可以明显看出,任何全局优化问题的难度都必然编码在平均场近似的速率中,对此我们提供了一种新的概率定量估计。这些结果的结合使我们能够获得数值CBO方法的概率全局收敛保证。
英文摘要:
In this paper, we study consensus-based optimization (CBO), which is a multi-agent metaheuristic derivative-free optimization method that can globally minimize nonconvex nonsmooth functions and is amenable to theoretical analysis. Based on an experimentally supported intuition that, on average, CBO performs a gradient descent of the squared Euclidean distance to the global minimizer, we devise a novel technique for proving the convergence to the global minimizer in mean-field law for a rich class of objective functions. The result unveils internal mechanisms of CBO that are responsible for the success of the method. In particular, we prove that CBO performs a convexification of a large class of optimization problems as the number of optimizing agents goes to infinity. Furthermore, we improve prior analyses by requiring mild assumptions about the initialization of the method and by covering objectives that are merely locally Lipschitz continuous. As a core component of this analysis, we establish a quantitative nonasymptotic Laplace principle, which may be of independent interest. From the result of CBO convergence in mean-field law, it becomes apparent that the hardness of any global optimization problem is necessarily encoded in the rate of the mean-field approximation, for which we provide a novel probabilistic quantitative estimate. The combination of these results allows to obtain probabilistic global convergence guarantees of the numerical CBO method.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。