任意采样的L-SVRG和L-喀秋莎
L-SVRG and L-Katyusha with Arbitrary Sampling
摘要：
我们开发和分析了一个新的家庭{ \em非加速和加速无环方差缩减方法}有限和最优化问题。我们的收敛性分析依赖于一个新的预期平滑条件的方差的随机梯度估计的上界由一个常数倍的距离函数。这使我们能够轻松处理{ \em任意采样方案}以及非凸的情况。我们进行了深入的估计，这些预期的平滑参数，并提出新的重要性采样，允许{ \em线性加速}时，预期的小批量大小是在一定范围内。此外，这些预期的平滑参数和预期的可分离过逼近（ESO）之间的连接建立，这使我们能够利用数据稀疏性以及。我们的结果恢复作为特殊情况下，最近提出的无环SVRG和无环卡秋莎。
英文摘要：
We develop and analyze a new family of {\em nonaccelerated and accelerated loopless variance-reduced methods} for finite sum optimization problems. Our convergence analysis relies on a novel expected smoothness condition which upper bounds the variance of the stochastic gradient estimation by a constant times a distance-like function. This allows us to handle with ease {\em arbitrary sampling schemes} as well as the nonconvex case. We perform an in-depth estimation of these expected smoothness parameters and propose new importance samplings which allow {\em linear speedup} when the expected minibatch size is in a certain range. Furthermore, a connection between these expected smoothness parameters and expected separable overapproximation (ESO) is established, which allows us to exploit data sparsity as well. Our results recover as special cases the recently proposed loopless SVRG and loopless Katyusha.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
