arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1603.09649math.OC
收藏

随机块BFGS:从数据中挤出更多曲率

Stochastic Block BFGS: Squeezing More Curvature out of Data

Robert M. Gower, Donald Goldfarb, Peter Richtárik

摘要:

我们提出了一种新的有限记忆随机块BFGS更新,将丰富的曲率信息的随机逼近方法。在我们的方法中,由它维护的逆黑塞矩阵的估计在每次迭代时使用黑塞矩阵的草图来更新,即,一个随机生成的压缩形式的海森。我们提出了几种绘制策略,提出了一种新的拟牛顿法,使用随机块BFGS更新结合方差减少方法SVRG计算批量随机梯度,并证明了所产生的方法的线性收敛。大规模的逻辑斯谛回归问题的数值试验表明,我们的方法是更强大的,大大优于当前国家的最先进的方法。

英文摘要:

We propose a novel limited-memory stochastic block BFGS update for incorporating enriched curvature information in stochastic approximation methods. In our method, the estimate of the inverse Hessian matrix that is maintained by it, is updated at each iteration using a sketch of the Hessian, i.e., a randomly generated compressed form of the Hessian. We propose several sketching strategies, present a new quasi-Newton method that uses stochastic block BFGS updates combined with the variance reduction approach SVRG to compute batch stochastic gradients, and prove linear convergence of the resulting method. Numerical tests on large-scale logistic regression problems reveal that our method is more robust and substantially outperforms current state-of-the-art methods.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑