arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2009.04103math.OC
收藏

分布式网络实时学习

Distributed Networked Real-time Learning

Alfredo Garcia, Luochao Wang, Jeff Huang, Lingzhou Hong

摘要:

许多机器学习算法都是在数据集已经以批量形式可用的假设下开发的。然而,在许多应用领域中,数据只能通过不同地理位置的计算节点按顺序随时间推移而可用。在本文中,我们考虑的问题,学习模型时,流数据不能及时转移到一个单一的位置。在这种情况下,需要一个分布式的学习架构,依赖于互联的“本地”节点网络。我们提出了一个分布式的计划,其中每个本地节点实现随机梯度更新的基础上,本地数据流。为了确保鲁棒估计,使用网络正则化惩罚来维持模型集合中的内聚性度量。我们表明,合奏平均近似一个固定点和表征的程度,个别模型不同于合奏平均。我们将结果与联邦学习进行比较,得出结论,该方法对数据流(数据速率和估计质量)的异质性更具鲁棒性。我们通过基于卷积神经网络的深度学习模型在图像分类中的应用来说明结果。

英文摘要:

Many machine learning algorithms have been developed under the assumption that data sets are already available in batch form. Yet in many application domains data is only available sequentially overtime via compute nodes in different geographic locations. In this paper, we consider the problem of learning a model when streaming data cannot be transferred to a single location in a timely fashion. In such cases, a distributed architecture for learning relying on a network of interconnected "local" nodes is required. We propose a distributed scheme in which every local node implements stochastic gradient updates based upon a local data stream. To ensure robust estimation, a network regularization penalty is used to maintain a measure of cohesion in the ensemble of models. We show the ensemble average approximates a stationary point and characterize the degree to which individual models differ from the ensemble average. We compare the results with federated learning to conclude the proposed approach is more robust to heterogeneity in data streams (data rates and estimation quality). We illustrate the results with an application to image classification with a deep learning model based upon convolutional neural networks.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑