arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1901.08669cs.LGmath.OCstat.ML
收藏

任意采样的SAGA

SAGA with Arbitrary Sampling

King Abdullah University of Science and Technology(阿卜杜拉国王科技大学);The University of Hong Kong(香港大学);University of Edinburgh(爱丁堡大学);Moscow Institute of Physics and Technology(莫斯科物理技术学院)

Xu Qian, Zheng Qu, Peter Richtárik

摘要:

我们研究了最小化大量光滑函数的平均值的问题,这在训练监督学习模型中是至关重要的。在这种情况下,最著名的方法之一是SAGA算法。尽管对这个问题进行了多年的研究,但SAGA的通用版本--一个包括任意重要性采样和minimizing方案的版本--并不存在。我们补救这种情况,并提出了一个通用的和灵活的变体SAGA { \em任意采样}范例。我们进行迭代复杂性分析的方法,很大程度上可能是由于新的随机李雅普诺夫函数的建设。我们建立了光滑和强凸制度下的线性收敛速度,并在二次函数增长条件下(即,在不假设强凸性的情况下)。我们的速率匹配的原始-对偶方法石英的任意采样分析是可用的,这使得一个重要的一步缩小差距,我们的理解的复杂性的原始和对偶方法的有限和问题。

英文摘要:

We study the problem of minimizing the average of a very large number of smooth functions, which is of key importance in training supervised learning models. One of the most celebrated methods in this context is the SAGA algorithm. Despite years of research on the topic, a general-purpose version of SAGA---one that would include arbitrary importance sampling and minibatching schemes---does not exist. We remedy this situation and propose a general and flexible variant of SAGA following the {\em arbitrary sampling} paradigm. We perform an iteration complexity analysis of the method, largely possible due to the construction of new stochastic Lyapunov functions. We establish linear convergence rates in the smooth and strongly convex regime, and under a quadratic functional growth condition (i.e., in a regime not assuming strong convexity). Our rates match those of the primal-dual method Quartz for which an arbitrary sampling analysis is available, which makes a significant step towards closing the gap in our understanding of complexity of primal and dual methods for finite sum problems.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑