We introduce a new stochastic differential model for global optimization of nonconvex functions on compact hypersurfaces. The model is inspired by the stochastic Kuramoto-Vicsek system and belongs to the class of Consensus-Based Optimization methods. In fact, particles move on the hypersurface driven by a drift towards an instantaneous consensus point, computed as a convex combination of the particle locations weighted by the cost function according to Laplace's principle. The consensus point represents an approximation to a global minimizer. The dynamics is further perturbed by a random vector field to favor exploration, whose variance is a function of the distance of the particles to the consensus point. In particular, as soon as the consensus is reached, then the stochastic component vanishes. In this paper, we study the well-posedness of the model and we derive rigorously its mean-field approximation for large particle limit.
This paper is concerned with the large particle limit for the consensus-based optimization (CBO), which was postulated in the pioneering works [6,28]. In order to solve this open problem, we adapt a compactness argument by first proving the tightness of the empirical measures $\{μ^N\}_{N\geq 2}$ associated to the particle system and then verifying that the limit measure $μ$ is the unique weak solution to the mean-field CBO equation. Such results are extended to the model of particle swarm optimization (PSO).
In this paper we provide an analytical framework for investigating the efficiency of a consensus-based model for tackling global optimization problems. This work justifies the optimization algorithm in the mean-field sense showing the convergence to the global minimizer for a large class of functions. Theoretical results on consensus estimates are then illustrated by numerical simulations where variants of the method including nonlinear diffusion are introduced.
Uniform-in-time propagation of chaos for Consensus-Based Optimization
Institute of Applied Analysis, Ulm University（乌尔姆大学应用分析研究所）；Department of Computing and Mathematical Sciences, Caltech（加州理工学院计算与数学科学系）；MATHERIALS team, Inria Paris（法国国家信息与自动化研究所巴黎 MATHERIALS 团队）；CERMICS, École des Ponts（巴黎路桥学校 CERMICS）
For algorithms based on interacting particle systems that admit a mean-field description, convergence analysis is often more accessible at the mean-field level. In order to transfer convergence results obtained at the mean-field level to the finite ensemble size setting, it is desirable to show that the particle dynamics converge in an appropriate sense to the corresponding mean-field dynamics. In this paper, we prove quantitative mean-field limit results for two related interacting particle systems: Consensus-Based Optimization and Consensus-Based Sampling. Our approach requires a generalizat…
Technical University of Munich（慕尼黑工业大学）；University of Calgary（卡尔加里大学）；University of Ferrara（费拉拉大学）
We investigate the implementation of a new stochastic Kuramoto-Vicsek-type model for global optimization of nonconvex functions on the sphere. This model belongs to the class of Consensus-Based Optimization. In fact, particles move on the sphere driven by a drift towards an instantaneous consensus point, which is computed as a convex combination of particle locations, weighted by the cost function according to Laplace's principle, and it represents an approximation to a global minimizer. The dynamics is further perturbed by a random vector field to favor exploration, whose variance is a functi…
An interacting particle consensus method for constrained global optimization
University of Oxford（牛津大学）；Shanghai Jiao Tong University（上海交通大学）；University of California, San Diego（加州大学圣迭戈分校）；University of California, Los Angeles（加州大学洛杉矶分校）
Regularity and positivity of solutions of the Consensus-Based Optimization equation: unconditional global convergence
Technical University of Munich, School of Computation, Information and Technology（慕尼黑工业大学计算、信息和技术学院）；Munich Center for Machine Learning（慕尼黑机器学习中心）；Munich Data Science Institute（慕尼黑数据科学研究所）
Leveraging Memory Effects and Gradient Information in Consensus-Based Optimization: On Global Convergence in Mean-Field Law
Technical University of Munich, School of Computation, Information and Technology, Department of Mathematics（慕尼黑工业大学计算、信息与技术学院数学系）；Munich Center for Machine Learning（慕尼黑机器学习中心）
