arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2207.05912math.OC
收藏

关于非单调梯度法的$R$ -线性收敛的注记

A note on $R$-linear convergence of nonmonotone gradient methods

Xinrui Li, Yakui Huang

摘要:

非单调梯度法通常比单调梯度法有更好的性能,特别是在无约束二次优化问题上。然而,已知的单调方法的收敛速度往往比它的非单调变体。为了缩小非单调梯度法的理论与实践之间的差距,我们引入了一个性质来分析大量的梯度法的收敛性。我们证明了任何梯度法使用的步长满足的性质将收敛$R$ -线性的速度为$1-λ_1/M_1$,其中$λ_1$是Hessian矩阵的最小特征值和$M_1$是逆步长的上界。我们的结果表明,现有的许多非单调方法的收敛速度可以提高到$1-1/κ$与$κ$是相关的条件数。

英文摘要:

Nonmonotone gradient methods generally perform better than their monotone counterparts especially on unconstrained quadratic optimization. However, the known convergence rate of the monotone method is often much better than its nonmonotone variant. With the aim of shrinking the gap between theory and practice of nonmonotone gradient methods, we introduce a property for convergence analysis of a large collection of gradient methods. We prove that any gradient method using stepsizes satisfying the property will converge $R$-linearly at a rate of $1-λ_1/M_1$, where $λ_1$ is the smallest eigenvalue of Hessian matrix and $M_1$ is the upper bound of the inverse stepsize. Our results indicate that the existing convergence rates of many nonmonotone methods can be improved to $1-1/κ$ with $κ$ being the associated condition number.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑