arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1811.09469stat.COmath.OCstat.ML
收藏

随机无梯度非凸优化问题的并行序贯蒙特卡罗

Parallel sequential Monte Carlo for stochastic gradient-free nonconvex optimization

Ömer Deniz Akyildiz, Dan Crisan, Joaquín Míguez

摘要:

我们介绍和分析了一个并行的序贯蒙特卡罗方法的数值解的优化问题,涉及到最小化的成本函数,包括许多单独的组件的总和。该方案是一个随机零阶优化算法,只需要评估成本函数的组件的小子集的能力。它可以被描述为一组采样器,这些采样器生成若干概率测度序列的粒子近似。这些措施被构造在这样一种方式,它们有相关的概率密度函数,其全球最大值与全球最小值的原始成本函数相一致。该算法选择性能最好的采样器,并使用它来近似成本函数的全局最小值。我们证明了分析估计收敛到一个全局最小值的成本函数几乎肯定,并提供明确的收敛速度生成的蒙特卡罗样本的数量和搜索空间的维度。我们表明,通过数值例子,该算法可以解决成本函数与多个极小值或广泛的“平坦”区域,这是很难最小化使用基于梯度的技术。

英文摘要:

We introduce and analyze a parallel sequential Monte Carlo methodology for the numerical solution of optimization problems that involve the minimization of a cost function that consists of the sum of many individual components. The proposed scheme is a stochastic zeroth order optimization algorithm which demands only the capability to evaluate small subsets of components of the cost function. It can be depicted as a bank of samplers that generate particle approximations of several sequences of probability measures. These measures are constructed in such a way that they have associated probability density functions whose global maxima coincide with the global minima of the original cost function. The algorithm selects the best performing sampler and uses it to approximate a global minimum of the cost function. We prove analytically that the resulting estimator converges to a global minimum of the cost function almost surely and provide explicit convergence rates in terms of the number of generated Monte Carlo samples and the dimension of the search space. We show, by way of numerical examples, that the algorithm can tackle cost functions with multiple minima or with broad "flat" regions which are hard to minimize using gradient-based techniques.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑