Katalyst：求解具有大条件数的非凸问题的Boosting凸Katayusha
Katalyst: Boosting Convex Katayusha for Non-Convex Problems with a Large Condition Number
摘要：
针对一类非凸优化问题，提出了一种新的基于SVRG的加速随机算法，该算法的目标函数由$n$光滑函数和非光滑凸函数组成.我们的主要目标是提高收敛的SVRG风格的随机算法下设置一个大的条件数$c$的平稳点-平滑常数和负曲率常数之间的比率。该算法实现了最好的梯度复杂度时，$c\geq Ω(n)$，这是以前实现的SAGA风格的加速随机算法。与SAGA风格的加速随机算法相比，该算法继承了以往SVRG风格算法的优点，具有较低的内存开销，因而更具有实用性。与以往的研究相比，SVRG风格的随机算法，我们的理论提供了更强的结果方面（i）降低梯度复杂度下的一个大的条件数;（ii）的收敛性被证明为一个采样阶段平均的解决方案，从所有阶段平均的解决方案，增加采样概率，而不是均匀采样的解决方案，在所有迭代。
英文摘要：
In this paper, we propose a new SVRG-style acceleated stochastic algorithm for solving a family of non-convex optimization problems whose objective consists of a sum of $n$ smooth functions and a non-smooth convex function. Our major goal is to improve the convergence of SVRG-style stochastic algorithms to stationary points under a setting with a large condition number $c$ - the ratio between the smoothness constant and the negative curvature constant. The proposed algorithm achieves the best known gradient complexity when $c\geq Ω(n)$, which was achieved previously by a SAGA-style accelerated stochastic algorithm. Compared with the SAGA-style accelerated stochastic algorithm, the proposed algorithm is more practical due to its low memory cost that is inherited from previous SVRG-style algorithms. Compared with previous studies on SVRG-style stochastic algorithms, our theory provides much stronger results in terms of (i) reduced gradient complexity under a large condition number; and (ii) that the convergence is proved for a sampled stagewise averaged solution that is selected from all stagewise averaged solutions with increasing sampling probabilities instead of for a uniformly sampled solutions across all iterations.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
