假设增长界限,一般持有人平滑的收敛速度遵循专门利率
General Holder Smooth Convergence Rates Follow From Specialized Rates Assuming Growth Bounds
摘要:
通常在光滑和非光滑最优化的一阶方法的分析中,假设存在增长/误差界或KL条件有利于更强的收敛性分析。因此,对于一般情况和增长受限情况,通常需要单独分析。我们给出的元定理推导一般的收敛速度从那些假设的增长下界。将这个简单但概念上强大的工具应用于邻近点、次梯度、束、对偶平均、梯度下降、Frank-Wolfe和通用加速方法,可以立即从其专用速度恢复其已知的一般凸优化问题的收敛速度。新的收敛结果如下束方法,双平均,和弗兰克-沃尔夫。我们的结果可以提升任何基于Hölder连续梯度和Hölder增长边界的速率。此外,我们的理论提供了简单的证明,最佳收敛下界下的Hölder增长教科书的例子没有增长的界限。
英文摘要:
Often in the analysis of first-order methods for both smooth and nonsmooth optimization, assuming the existence of a growth/error bound or KL condition facilitates much stronger convergence analysis. Hence separate analysis is typically needed for the general case and for the growth bounded cases. We give meta-theorems for deriving general convergence rates from those assuming a growth lower bound. Applying this simple but conceptually powerful tool to the proximal point, subgradient, bundle, dual averaging, gradient descent, Frank-Wolfe, and universal accelerated methods immediately recovers their known convergence rates for general convex optimization problems from their specialized rates. New convergence results follow for bundle methods, dual averaging, and Frank-Wolfe. Our results can lift any rate based on Hölder continuous gradients and Hölder growth bounds. Moreover, our theory provides simple proofs of optimal convergence lower bounds under Hölder growth from textbook examples without growth bounds.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。