arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2007.06799stat.MLcs.LGmath.STstat.TH
收藏

贝叶斯学习的分散方法

A Decentralized Approach to Bayesian Learning

CCDC Army Research Laboratory(CCDC陆军研究实验室);Oklahoma State University(俄克拉荷马州立大学);Booz Allen Hamilton(博思艾伦汉密尔顿公司)

Anjaly Parayil, He Bai, Jemin George, Prudhvi Gurram

摘要:

受分散式机器学习方法的启发,我们提出了一种协作贝叶斯学习算法,该算法采用非凸设置中分散式Langevin动态的形式。我们的分析表明,初始KL之间的马尔可夫链和目标后验分布的分歧是指数下降,而错误的贡献,从加性噪声的整体KL发散是减少在多项式时间。我们进一步表明,多项式项的经验加速与代理的数量,并提供充分的条件,随时间变化的步长,以保证收敛到所需的分布。所提出的算法的性能进行了评估各种各样的机器学习任务。实证结果表明,与集中式设置相当大的改善收敛速度的个人代理与本地可用的数据的性能。

英文摘要:

Motivated by decentralized approaches to machine learning, we propose a collaborative Bayesian learning algorithm taking the form of decentralized Langevin dynamics in a non-convex setting. Our analysis show that the initial KL-divergence between the Markov Chain and the target posterior distribution is exponentially decreasing while the error contributions to the overall KL-divergence from the additive noise is decreasing in polynomial time. We further show that the polynomial-term experiences speed-up with number of agents and provide sufficient conditions on the time-varying step-sizes to guarantee convergence to the desired distribution. The performance of the proposed algorithm is evaluated on a wide variety of machine learning tasks. The empirical results show that the performance of individual agents with locally available data is on par with the centralized setting with considerable improvement in the convergence rate.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑