arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2407.02092cond-mat.dis-nncond-mat.stat-mech
收藏

通过边缘最优解的统计特性对复杂优化进行与算法无关的界限分析

Algorithm-independent bounds on complex optimization through the statistics of marginal optima

Jaron Kent-Dobias

AI总结:

针对超大量最优解导致优化算法易陷入停滞的问题,利用随机景观中边缘最优解的统计特性推导出与算法无关的优化界限,并通过梯度下降等数值实验验证了其有效性。

摘要:

优化旨在寻找函数中的极值点。当存在超大量的最优解时,优化算法极易陷入停滞。在此条件下,通用算法倾向于找到边缘最优解,这些解具有许多近乎平坦的方向。在配套论文中,我们引入了一种计算随机景观中边缘最优解数量的技术。在此,我们利用该技术计算出的边缘最优解统计特性,基于算法将绝大多数情况下仅在存在边缘最优解处陷入停滞这一简单原理,推导出了优化的通用界限。我们在一个简单的非高斯问题上验证了该想法,即在紧致空间上最大化随机函数平方和。使用梯度下降和广义近似消息传递算法进行的数值实验均落在预期界限之内。

英文摘要:

Optimization seeks extremal points in a function. When there are superextensively many optima, optimization algorithms are liable to get stuck. Under these conditions, generic algorithms tend to find marginal optima, which have many nearly flat directions. In a companion paper, we introduce a technique to count marginal optima in random landscapes. Here, we use the statistics of marginal optima calculated using this technique to produce generic bounds on optimization, based on the simple principle that algorithms will overwhelmingly tend to get stuck only where marginal optima are found. We demonstrate the idea on a simple non-Gaussian problem of maximizing the sum of squared random functions on a compact space. Numeric experiments using both gradient descent and generalized approximate message passing algorithms fall inside the expected bounds.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑