arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1706.05671math.OC
收藏

Nesterov加速梯度法在次临界情况下收敛速率$α\leq 3$

Rate of convergence of the Nesterov accelerated gradient method in the subcritical case $α\leq 3$

Hedy Attouch, Zaki Chbani, Hassan Riahi

摘要:

在Hilbert空间中,给定$\mathcal H$为凸连续可微函数,$α$为正参数,考虑具有渐近消失阻尼\begin{equation*} \mbox{(AVD)}_α \quad \quad \ddot{x}(t) + \fracα{t} \dot{x}(t) + \nabla Φ(x(t)) =0. \end{equation*}的惯性系统。根据$ α$关于3的值,给出了由$\mbox{(AVD)}_α$生成的轨迹作为$t \to + \infty$的收敛性质以及相应算法的迭代过程。我们的主要结果涉及的亚临界情况下$α\leq 3$,我们表明,$Φ(x(t))-\min Φ= \mathcal O (t^{-\frac{2}{3}α})$。然后,我们研究的轨迹收敛到最优解。作为一个新的结果,在一维框架下,对于临界值$α= 3 $,我们证明了轨迹的收敛性,而不对凸函数$Φ$作任何限制性假设。在本文的第二部分,我们研究了相关的前向后惯性算法的收敛性。他们的目标是解决结构化凸极小化问题的形式$\min \left\lbrace Θ:= Φ+ Ψ\right\rbrace$,$Φ$光滑和$Ψ$非光滑。连续动力学作为本研究的指导方针。对于迭代序列$(x_k)$,我们获得了类似的收敛速率:对于$α\leq 3$,我们对所有$p <\frac{2α}{3}$都有$Θ(x_k)-\min Θ= \mathcal O (k^{-p})$,对于$α> 3$ \ $Θ(x_k)-\min Θ= o (k^{-2})$。我们得出结论表明,结果是强大的外部扰动这项研究。

英文摘要:

In a Hilbert space setting $\mathcal H$, given $Φ: \mathcal H \to \mathbb R$ a convex continuously differentiable function, and $α$ a positive parameter, we consider the inertial system with Asymptotic Vanishing Damping \begin{equation*} \mbox{(AVD)}_α \quad \quad \ddot{x}(t) + \fracα{t} \dot{x}(t) + \nabla Φ(x(t)) =0. \end{equation*} Depending on the value of $ α$ with respect to 3, we give a complete picture of the convergence properties as $t \to + \infty$ of the trajectories generated by $\mbox{(AVD)}_α$, as well as iterations of the corresponding algorithms. Our main result concerns the subcritical case $α\leq 3$, where we show that $Φ(x(t))-\min Φ= \mathcal O (t^{-\frac{2}{3}α})$. Then we examine the convergence of trajectories to optimal solutions. As a new result, in the one-dimensional framework, for the critical value $α= 3 $, we prove the convergence of the trajectories without any restrictive hypothesis on the convex function $Φ$. In the second part of this paper, we study the convergence properties of the associated forward-backward inertial algorithms. They aim to solve structured convex minimization problems of the form $\min \left\lbrace Θ:= Φ+ Ψ\right\rbrace$, with $Φ$ smooth and $Ψ$ nonsmooth. The continuous dynamics serves as a guideline for this study. We obtain a similar rate of convergence for the sequence of iterates $(x_k)$: for $α\leq 3$ we have $Θ(x_k)-\min Θ= \mathcal O (k^{-p})$ for all $p <\frac{2α}{3}$ , and for $α> 3$ \ $Θ(x_k)-\min Θ= o (k^{-2})$ . We conclude this study by showing that the results are robust with respect to external perturbations.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑