AI 中文总结
该研究改进了IGAHD的李雅普诺夫分析中的参数条件,使其适用于更小的步长,还针对凸二次函数得到了更大的参数区域,并提出了关于非二次函数的待研究问题。
AI 中文摘要
在最近的一篇论文中,Attouch、Chbani、Fadili和Riahi提出了带 Hessian 驱动阻尼的惯性梯度算法,记为 IGAHD。在条件 $0\leq\beta<2\sqrt{s}$(其中 $\beta$ 是 Hessian 驱动阻尼参数,$s>0$ 是梯度步长)下,他们的李雅普诺夫分析表明,当 $\beta>0$ 时,目标函数存在加速估计,且梯度具有加权可和性。对于任意固定的 $\beta>0$,该条件排除了足够小的步长 $s>0$。本注通过保留之前被下界替代的两个系数,改进了该李雅普诺夫分析中的一个估计,得到了如下限制性更弱的充分条件:$$0<\beta L\sqrt{s}<1+\sqrt{1+sL(1-sL)}$$,其中 $L>0$ 是梯度目标函数的 Lipschitz 常数,且 $0<s\leq1/L$。特别地,最简单的条件 $0<\beta <\frac{2}{L\sqrt{s}}$ 是充分的。对于固定的 $\beta>0$ 和 $L>0$,该新条件对足够小的 $s>0$ 成立。因此,在这个新的改进充分条件下,原论文中的结论仍然有效。对于有限维中的凸二次函数,单独的谱分析得到了更大的参数区域,它给出了极限模态矩阵 Schur 稳定性的充要条件,以及目标残差和梯度的几何衰减。该模态分析提出了一个问题:对于非二次目标函数,不同的李雅普诺夫函数是否能让我们恢复该扩展谱域的部分(或全部)。
英文摘要
In a recent paper, Attouch, Chbani, Fadili and Riahi introduced the inertial gradient algorithm with Hessian-driven daming, called (IGAHD). Under the condition $0\leqβ<2\sqrt{s}$, where $β$ is the Hessian driven damping parameter and $s>0$ is the gradient step size, their Lyapunov analysis shows an accelerated estimate of the objective function as well as a weighted summability of the gradient for the case $β>0$. For any fixed value of $β>0$, this condition excluded sufficiently small step sizes $s>0$. The present note refines one of the estimates in this Lyapunov analysis by retaining two coefficients that were previously replaced by lower bounds. This leads to the following less restrictive and sufficient condition: $$0<βL\sqrt{s}<1+\sqrt{1+sL(1-sL)},$$ where $L>0$ is the constant Lipschitz of the gradient objective function and $0<s\leq1/L$. In particular, the simplest condition $0<β<\frac{2}{L\sqrt{s}}$ is sufficient. For fixed values of $β>0$ and $L>0$, this new condition is satisfied for sufficiently small $s>0$. Consequently, under this new refined sufficient condition, the conclusions in the original paper stay valid. For convex quadratic functions in finite dimensions, a separate spectral analysis leads to a larger region parameters. It gives a necessary and sufficient condition for Schur stability of the limit modal matrices, as well as the geometrical decay of the objective residuals and the gradients. This modal analysis raises the question of whether a different Lyapunov function would allow us to recover part (or all) of this extended spectral domain for non-quadratic objective functions.