MISO正在以更好的证明和价格卷土重来
MISO is Making a Comeback With Better Proofs and Rates
摘要：
MISO，也被称为MISO，是最早发现的随机方差缩减方法之一，但它的流行程度相当低。其最初的分析受到所谓的大数据假设的极大限制。虽然在随后的工作中使用负动量取消了这个假设，但这引入了一个新的参数，并且需要强凸性和光滑常数的知识，这在实践中很少可能。我们恢复的方法，通过引入一个新的变种，只需要平滑常数，没有任何额外的参数。此外，当删除强凸常数的步长，我们提出了一个新的分析方法，不再使用的假设，每个组件是强凸的。这使得我们也获得迄今未知的非凸收敛MISO。为了使所提出的方法在实践中有效，我们推导出minibatch边界与任意均匀采样，导致线性加速时，预期的minibatch大小是在一定范围内。我们的数值实验表明，MISO是SAGA和SVRG的有力竞争者，有时在真实数据集上的表现优于它们。
英文摘要：
MISO, also known as Finito, was one of the first stochastic variance reduced methods discovered, yet its popularity is fairly low. Its initial analysis was significantly limited by the so-called Big Data assumption. Although the assumption was lifted in subsequent work using negative momentum, this introduced a new parameter and required knowledge of strong convexity and smoothness constants, which is rarely possible in practice. We rehabilitate the method by introducing a new variant that needs only smoothness constant and does not have any extra parameters. Furthermore, when removing the strong convexity constant from the stepsize, we present a new analysis of the method, which no longer uses the assumption that every component is strongly convex. This allows us to also obtain so far unknown nonconvex convergence of MISO. To make the proposed method efficient in practice, we derive minibatching bounds with arbitrary uniform sampling that lead to linear speedup when the expected minibatch size is in a certain range. Our numerical experiments show that MISO is a serious competitor to SAGA and SVRG and sometimes outperforms them on real datasets.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
