基于受控贝叶斯空中聚集的异构数据联邦学习
Federated Learning from Heterogeneous Data via Controlled Bayesian Air Aggregation
摘要:
联邦学习(FL)是一种新兴的机器学习范式,用于在多个边缘设备上训练模型,这些边缘设备保存本地数据集,而无需显式交换数据。最近,已经提出空中(OTA)FL,以通过允许用户在多址接入信道(MAC)上同时发送他们的数据来减少带宽和能量消耗。然而,这种方法会导致信道噪声直接影响优化过程,这可能会降低训练模型的准确性。在本文中,我们联合利用本地权重的先验分布和信道分布,并开发了一个OTA FL算法的基础上贝叶斯方法的信号聚合。我们提出的算法,被称为贝叶斯空气聚合联合学习(BAAF),有效地减轻噪声和信道引起的衰落效应。为了处理用户数据的统计异质性,这是FL的第二个主要挑战,我们扩展了BAAF,允许用户进行适当的本地更新,并开发了受控贝叶斯空中聚合联合学习(COBAAF)算法。除了使用贝叶斯方法来平均通道输出之外,COBAAF还使用校正项的明智设计来控制局部更新中的漂移。我们使用BAAF和COBAAF在嘈杂和异构的环境中学习的全局模型的收敛性进行了分析,显示了它们实现与无错误信道相似的收敛速度的能力。仿真结果表明,改进的收敛性BAAF和COBAAF在机器学习任务中比现有的算法。
英文摘要:
Federated learning (FL) is an emerging machine learning paradigm for training models across multiple edge devices holding local data sets, without explicitly exchanging the data. Recently, over-the-air (OTA) FL has been suggested to reduce the bandwidth and energy consumption, by allowing the users to transmit their data simultaneously over a Multiple Access Channel (MAC). However, this approach results in channel noise directly affecting the optimization procedure, which may degrade the accuracy of the trained model. In this paper we jointly exploit the prior distribution of local weights and the channel distribution, and develop an OTA FL algorithm based on a Bayesian approach for signal aggregation. Our proposed algorithm, dubbed Bayesian Air Aggregation Federated learning (BAAF), is shown to effectively mitigate noise and fading effects induced by the channel. To handle statistical heterogeneity of users data, which is a second major challenge in FL, we extend BAAF to allow for appropriate local updates by the users and develop the Controlled Bayesian Air Aggregation Federated-learning (COBAAF) algorithm. In addition to using a Bayesian approach to average the channel output, COBAAF controls the drift in local updates using a judicious design of correction terms. We analyze the convergence of the learned global model using BAAF and COBAAF in noisy and heterogeneous environment, showing their ability to achieve a convergence rate similar to that achieved over error-free channels. Simulation results demonstrate the improved convergence of BAAF and COBAAF over existing algorithms in machine learning tasks.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。