arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2412.05604math.OCmath.STstat.TH
收藏

基于策略性大数定律的优化方法

Optimization via Strategic Law of Large Numbers

Cowles Foundation for Research in Economics, Yale University(耶鲁大学科尔斯经济学研究基金会);Zhongtai Securities Institute for Financial Studies, Shandong University(山东大学中泰证券金融研究院);Department of Economics, University of Pennsylvania(宾夕法尼亚大学经济学系);School of Mathematics and Statistics, Xi’an Jiaotong University(西安交通大学数学与统计学院);School of Statistics and Mathematics, Shandong University of Finance and Economics(山东财经大学统计与数学学院)

Xiaohong Chen, Zengjing Chen, Wayne Yuan Gao, Xiaodong Yan, Guodong Zhang

AI总结:

本文提出策略性蒙特卡洛优化(SMCO)算法,基于偏梯度符号进行逐坐标双臂决策,无需求解偏微分方程,仅用少量起始点和迭代次数即可在高维非凸优化中稳健收敛到全局最优解。

摘要:

本文提出了一个用于有界矩形区域中连续函数全局优化的统一框架。具体而言,我们证明:(1)在双臂决策模型的最优策略下,样本均值在策略性大数定律下收敛到全局最优解;(2)基于抛物型偏微分方程解的符号策略是渐近最优的。受此结果启发,我们提出了一类策略性蒙特卡洛优化(Strategic Monte Carlo Optimization,SMCO)算法,该算法使用一种简单策略,基于被优化函数偏梯度的符号进行逐坐标的双臂决策(无需求解偏微分方程)。虽然这种简单策略通常不是最优的,但我们证明它足以使我们的SMCO算法从单个起始点收敛到局部最优解,并在起始点集合不断增大的条件下收敛到全局最优解。数值研究证明了我们的SMCO算法适用于全局优化,并展示了我们的理论框架和实用方法的前景。对于大量具有挑战性优化景观的测试函数(包括具有平方损失和铰链损失的ReLU神经网络),我们的SMCO算法仅使用少量起始点(维度最高为1000时最多100个)和较小的最大迭代次数(200次),就能准确且稳健地收敛到全局最大值。事实上,我们的算法优于许多最先进的全局优化器,以及使用与我们相同起始点集合增强的局部算法。

英文摘要:

This paper proposes a unified framework for the global optimization of a continuous function in a bounded rectangular domain. Specifically, we show that: (1) under the optimal strategy for a two-armed decision model, the sample mean converges to a global optimizer under the Strategic Law of Large Numbers, and (2) a sign-based strategy built upon the solution of a parabolic PDE is asymptotically optimal. Motivated by this result, we propose a class of {\bf S}trategic {\bf M}onte {\bf C}arlo {\bf O}ptimization (SMCO) algorithms, which uses a simple strategy that makes coordinate-wise two-armed decisions based on the signs of the partial gradient of the original function being optimized over (without the need of solving PDEs). While this simple strategy is not generally optimal, we show that it is sufficient for our SMCO algorithm to converge to local optimizer(s) from a single starting point, and to global optimizers under a growing set of starting points. Numerical studies demonstrate the suitability of our SMCO algorithms for global optimization, and illustrate the promise of our theoretical framework and practical approach. For a wide range of test functions with challenging optimization landscapes (including ReLU neural networks with square and hinge loss), our SMCO algorithms converge to the global maximum accurately and robustly, using only a small set of starting points (at most 100 for dimensions up to 1000) and a small maximum number of iterations (200). In fact, our algorithms outperform many state-of-the-art global optimizers, as well as local algorithms augmented with the same set of starting points as ours.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑