基于灰色关联分析的非独立同分布数据联邦学习客户端选择优化
Optimization of Federated Learning's Client Selection for Non-IID Data Based on Grey Relational Analysis
AI总结:
针对非IID数据与设备异构下随机客户端选择导致训练效率下降的问题,提出基于GRA、综合计算资源、训练损失和权重发散的客户端选择方法,并在AWS实验中证明其提升准确率并降低等待时间。
摘要:
联邦学习(FL)是一种为隐私敏感数据应用设计的新型分布式学习框架。在不共享数据的情况下,FL在各个设备上训练本地模型,并由服务器通过执行模型聚合来构建全局模型。然而,为降低通信成本,每一轮训练的参与者都是随机选择的,这在数据和设备异构性下显著降低了训练效率。为解决这一问题,本文提出一种新方法,综合考虑数据分布和设备计算资源,为每一轮训练选择客户端。所提方法基于灰色关联分析(GRA)理论进行客户端选择,同时考虑每个客户端的可用计算资源、训练损失和权重发散。为检验该方法的可用性,我们使用Python的TensorFlow库在Amazon Web Services(AWS)上实现了这一贡献。我们通过改变学习率、网络规模、所选客户端数量以及客户端选择轮次,在不同设置下评估算法性能。评估结果表明,与最先进的方法federated averaging和Pow-d相比,所提算法在测试准确率和客户端平均等待时间方面显著提升了性能。
英文摘要:
Federated learning (FL) is a novel distributed learning framework designed for applications with privacy-sensitive data. Without sharing data, FL trains local models on individual devices and constructs the global model on the server by performing model aggregation. However, to reduce the communication cost, the participants in each training round are randomly selected, which significantly decreases the training efficiency under data and device heterogeneity. To address this issue, in this paper, we introduce a novel approach that considers the data distribution and computational resources of devices to select the clients for each training round. Our proposed method performs client selection based on the Grey Relational Analysis (GRA) theory by considering available computational resources for each client, the training loss, and weight divergence. To examine the usability of our proposed method, we implement our contribution on Amazon Web Services (AWS) by using the TensorFlow library of Python. We evaluate our algorithm's performance in different setups by varying the learning rate, network size, the number of selected clients, and the client selection round. The evaluation results show that our proposed algorithm enhances the performance significantly in terms of test accuracy and the average client's waiting time compared to state-of-the-art methods, federated averaging and Pow-d.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。