arXiv 1303.46452013-03math.OCcs.ITcs.NAmath.ITmath.NA

凸最小化的梯度方法：在更弱条件下获得更好的速率

Gradient methods for convex minimization: better rates under weaker conditions

National University of Defense Technology（国防科技大学）；Rice University（莱斯大学）

Hui Zhang, Wotao Yin

备注：: 20 pages, 4 figures, typos are corrected, Theorem 2 is new

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文证明在比通常更弱的条件下，普通和加速梯度方法仍能达到已知的复杂度，并利用松弛条件得到更优的复杂度界。

摘要：

梯度方法在最小化凸可微函数时的收敛行为是凸优化中的核心问题之一。本文表明，在比通常接受的条件更弱的条件下，可以实现它们众所周知的复杂度。我们将常见的梯度Lipschitz连续条件和强凸条件放松为仅在某些线段上成立的条件。具体地，假设对于每个$x\in\dom f$，$ abla f$在连接$x$和$x- rac{1}{R} abla f$的线段上是Lipschitz连续的，常数为$R$，我们分别建立了普通和加速梯度方法的复杂度$O( rac{R}{ε})$和$O(\sqrt{ rac{R}{ε}})$。然后，对于也满足割线不等式$\langle abla f(x), x- x^* angle \ge ν\|x-x^*\|^2$（对每个$x\in\dom f$及其到$f$的最小值集的投影$x^*$）的函数$f$，我们将复杂度改进为$O( rac{R}{ν}\log( rac{1}{ε}))$和$O(\sqrt{ rac{R}{ν}}\log( rac{1}{ε}))$。还证明了割线条件对于解误差的几何衰减是必要的。松弛条件不仅被更多函数满足，而且限制条件使得$R$更小、$ν$更大，从而得到更好的复杂度界。我们将这些结果应用于稀疏优化，并展示了一个更快的算法。

英文摘要：

The convergence behavior of gradient methods for minimizing convex differentiable functions is one of the core questions in convex optimization. This paper shows that their well-known complexities can be achieved under conditions weaker than the commonly accepted ones. We relax the common gradient Lipschitz-continuity condition and strong convexity condition to ones that hold only over certain line segments. Specifically, we establish complexities $O(\frac{R}ε)$ and $O(\sqrt{\frac{R}ε})$ for the ordinary and accelerate gradient methods, respectively, assuming that $\nabla f$ is Lipschitz continuous with constant $R$ over the line segment joining $x$ and $x-\frac{1}{R}\nabla f$ for each $x\in\dom f$. Then we improve them to $O(\frac{R}ν\log(\frac{1}ε))$ and $O(\sqrt{\frac{R}ν}\log(\frac{1}ε))$ for function $f$ that also satisfies the secant inequality $\ < \nabla f(x), x- x^*\ > \ge ν\|x-x^*\|^2$ for each $x\in \dom f$ and its projection $x^*$ to the minimizer set of $f$. The secant condition is also shown to be necessary for the geometric decay of solution error. Not only are the relaxed conditions met by more functions, the restrictions give smaller $R$ and larger $ν$ than they are without the restrictions and thus lead to better complexity bounds. We apply these results to sparse optimization and demonstrate a faster algorithm.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

AI总结：

摘要：

英文摘要：

相关论文