arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2203.01744cs.LGmath.OCstat.ML
收藏

非强凸最小二乘的加速SGD

Accelerated SGD for Non-Strongly-Convex Least Squares

EPFL(洛桑联邦理工学院)

Aditya Varre, Nicolas Flammarion

摘要:

考虑非强凸情形下最小二乘回归问题的随机逼近。我们提出了第一个实用的算法,实现了最佳的预测出错率的依赖于噪声的问题,$O(d/t)$,同时加速忘记的初始条件$O(d/t^2)$。我们的新算法是基于一个简单的修改的加速梯度下降。我们提供了收敛结果的平均值和最后一次的算法。为了描述这些新的界限的紧密性,我们提出了一个匹配的下限在无噪声设置,从而显示我们的算法的最优性。

英文摘要:

We consider stochastic approximation for the least squares regression problem in the non-strongly convex setting. We present the first practical algorithm that achieves the optimal prediction error rates in terms of dependence on the noise of the problem, as $O(d/t)$ while accelerating the forgetting of the initial conditions to $O(d/t^2)$. Our new algorithm is based on a simple modification of the accelerated gradient descent. We provide convergence results for both the averaged and the last iterate of the algorithm. In order to describe the tightness of these new bounds, we present a matching lower bound in the noiseless setting and thus show the optimality of our algorithm.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑