arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1908.06693math.OCcs.SYeess.SY
收藏

非凸问题的分布式随机梯度法及其在监督学习中的应用

Distributed Stochastic Gradient Method for Non-Convex Problems with Applications in Supervised Learning

Jemin George, Tao Yang, He Bai, Prudhvi Gurram

摘要:

在局部目标函数具有Lipschitz连续梯度和Hessian函数的二次连续可微的假设下,提出了一种求解非凸优化问题的分布式随机梯度下降算法.我们提供了充分条件的步长,保证所提出的算法的渐近均方收敛。我们将开发的算法应用到分布式监督学习问题,其中一组网络代理协作训练他们各自的神经网络来识别图像中的手写数字。结果表明,所有代理报告类似的性能,也是一个集中训练的神经网络的性能相媲美。数值结果还表明,所提出的分布式算法允许个别代理识别的数字,即使对应于所有数字的训练数据是不是本地提供给每个代理。

英文摘要:

We develop a distributed stochastic gradient descent algorithm for solving non-convex optimization problems under the assumption that the local objective functions are twice continuously differentiable with Lipschitz continuous gradients and Hessians. We provide sufficient conditions on step-sizes that guarantee the asymptotic mean-square convergence of the proposed algorithm. We apply the developed algorithm to a distributed supervised-learning problem, in which a set of networked agents collaboratively train their individual neural nets to recognize handwritten digits in images. Results indicate that all agents report similar performance that is also comparable to the performance of a centrally trained neural net. Numerical results also show that the proposed distributed algorithm allows the individual agents to recognize the digits even though the training data corresponding to all the digits is not locally available to each agent.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑