arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1805.01916math.OCstat.CO
收藏

非光滑随机逼近分析:微分包含方法

Analysis of nonsmooth stochastic approximation: the differential inclusion approach

Szymon Majewski, Błażej Miasojedow, Eric Moulines

摘要:

本文讨论了当被极小化的函数不是凸非光滑函数时,随机逼近的收敛性。我们表明,“平均极限”的收敛方法,导致光滑的问题,常微分方程的方法可以适应非光滑的情况。在适当的假设下,极限动力系统可以被证明是一个微分包含。我们的结果扩展了Benaim等人(2005)在这个方向上的早期工作,并提供了一个通用的框架来证明无约束和约束随机逼近问题的收敛性,无论是显式更新还是隐式更新。特别是,我们的研究结果使我们能够建立随机次梯度和近端随机梯度下降算法的收敛性,这些算法产生于一大类深度学习和高维统计推断,并具有稀疏诱导惩罚。

英文摘要:

In this paper we address the convergence of stochastic approximation when the functions to be minimized are not convex and nonsmooth. We show that the "mean-limit" approach to the convergence which leads, for smooth problems, to the ODE approach can be adapted to the non-smooth case. The limiting dynamical system may be shown to be, under appropriate assumption, a differential inclusion. Our results expand earlier works in this direction by Benaim et al. (2005) and provide a general framework for proving convergence for unconstrained and constrained stochastic approximation problems, with either explicit or implicit updates. In particular, our results allow us to establish the convergence of stochastic subgradient and proximal stochastic gradient descent algorithms arising in a large class of deep learning and high-dimensional statistical inference with sparsity inducing penalties.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑