arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2203.12199math.OCmath.CA
收藏

具有能量和动量的梯度算法的动态行为

Dynamic behavior for a gradient algorithm with energy and momentum

Hailiang Liu, Xuping Tian

AI总结:

本文提出结合能量与动量的梯度算法AGEM,通过高分辨率ODE系统分析其动态行为,证明解的一致有界性、收敛到临界点,并在Polyak-Łojasiewicz条件下建立线性收敛速率。

摘要:

本文研究了一种新颖的梯度算法AGEM,该算法同时利用能量和动量来解决一般的非凸优化问题。我们考察了AGEM算法的解的性质,包括一致有界性和收敛到临界点等方面。通过对一个高分辨率ODE系统的全面分析,研究了其动态行为。该ODE系统是非线性的,通过对离散格式取极限并通过对动量参数进行重缩放来保留动量效应而得到。本文强调了该ODE系统的全局适定性和解轨迹的时间渐近收敛性。此外,对于满足Polyak-Łojasiewicz条件的目标函数,我们建立了线性收敛速率。

英文摘要:

This paper investigates a novel gradient algorithm, AGEM, using both energy and momentum, for addressing general non-convex optimization problems. The solution properties of the AGEM algorithm, including aspects such as uniformly boundedness and convergence to critical points, are examined. The dynamic behavior is studied through a comprehensive analysis of a high-resolution ODE system. This ODE system, being nonlinear, is derived by taking the limit of the discrete scheme while preserving the momentum effect through a rescaling of the momentum parameter. The paper emphasizes the global well-posedness of the ODE system and the time-asymptotic convergence of solution trajectories. Furthermore, we establish a linear convergence rate for objective functions that adhere to the Polyak-Łojasiewicz condition.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑