arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2207.02750math.OC
收藏

随机凸优化的 SDE 视角

An SDE perspective on stochastic convex optimization

Rodrigo Maulen-Soto, Jalal Fadili, Hedy Attouch

AI总结:

本文以随机微分方程刻画噪声梯度流,证明凸优化轨迹几乎必然收敛,并给出凸、强凸及局部 Łojasiewicz 情形的期望收敛率,且推广至约束、非光滑及 Moreau 包络问题。

摘要:

我们分析了在随机误差下类梯度流的全局和局部行为,旨在解决带有噪声梯度输入的凸优化问题。我们首先研究无约束可微凸情形,使用一个随机微分方程,其中漂移项为目标函数梯度的负值,扩散项要么有界,要么平方可积。在此背景下,在梯度具有 Lipschitz 连续性的条件下,我们的第一个主要结果表明,目标函数和轨迹过程几乎必然收敛到目标函数的一个极小点。我们还提供了全面的复杂性分析,针对凸、强凸以及(局部)Łojasiewicz 情形,建立了若干新的期望意义下的逐点和遍历收敛率。后者涉及局部分析,具有挑战性,需要来自测度论的非平凡论证。随后,我们将研究扩展到约束情形,并更一般地扩展到某些非光滑情形。我们表明,通过将目标函数的梯度替换为一个余强制单调算子,我们的若干结果可以得到自然推广。这使得针对具有加性“光滑+非光滑”凸结构的优化问题获得类似收敛结果成为可能。最后,我们考虑了基于 Moreau 包络将结果扩展到非光滑优化的另一种方式。

英文摘要:

We analyze the global and local behavior of gradient-like flows under stochastic errors towards the aim of solving convex optimization problems with noisy gradient input. We first study the unconstrained differentiable convex case, using a stochastic differential equation where the drift term is minus the gradient of the objective function and the diffusion term is either bounded or square-integrable. In this context, under Lipschitz continuity of the gradient, our first main result shows almost sure convergence of the objective and the trajectory process towards a minimizer of the objective function. We also provide a comprehensive complexity analysis by establishing several new pointwise and ergodic convergence rates in expectation for the convex, strongly convex, and (local) Łojasiewicz case. The latter, which involves local analysis, is challenging and requires non-trivial arguments from measure theory. Then, we extend our study to the constrained case and more generally to certain nonsmooth situations. We show that several of our results have natural extensions obtained by replacing the gradient of the objective function by a cocoercive monotone operator. This makes it possible to obtain similar convergence results for optimization problems with an additively "smooth + non-smooth" convex structure. Finally, we consider another extension of our results to non-smooth optimization which is based on the Moreau envelope.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑