arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2107.08686cs.LGmath.OCstat.ML
收藏

改进的随机优化学习率

Improved Learning Rates for Stochastic Optimization

Renmin University of China(中国人民大学)

Shaojie Li, Pengwei Tang, Yong Liu

AI总结:

本文研究了随机优化中SGD和NAG算法的泛化性能,推导了改进的学习率理论,并通过数值实验验证了结果。

摘要:

随机优化是现代机器学习的核心。本文研究了两种经典随机优化算法:随机梯度下降(SGD)和Nesterov加速梯度(NAG)的泛化性能。我们为这两种算法建立了新的学习率,某些情况下提供了改进的保证,而在较弱的假设下实现了可比的速率。我们还提供了数值实验来支持理论。

英文摘要:

Stochastic optimization is a cornerstone of modern machine learning. This paper studies the generalization performance of two classical stochastic optimization algorithms: stochastic gradient descent (SGD) and Nesterov's accelerated gradient (NAG). We establish new learning rates for both algorithms, with improved guarantees in some settings or comparable rates under weaker assumptions in others. We also provide numerical experiments to support the theory.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑