arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2306.04207cs.DC
收藏

应对联邦学习中参与者异构性的资源感知聚类

Resource Aware Clustering for Tackling the Heterogeneity of Participants in Federated Learning

Rahul Mishra, Hari Prabhat Gupta, Garvit Banga

AI总结:

针对联邦学习参与者设备与网络资源异构导致训练延迟的问题,提出基于 Dunn 指数的资源感知聚类、参与者分配及主从知识蒸馏方法,并通过实验验证其有效性。

摘要:

联邦学习是一种训练框架,允许多个参与者协同训练共享模型,同时保护数据隐私并最小化通信开销。参与者设备和网络资源的异构性会延迟联邦学习中的训练与聚合。本文提出一种联邦学习方法,通过资源感知聚类来调控参与者之间的异构性。该方法首先由服务器收集参与者的设备和网络资源信息,随后执行资源感知聚类,并利用 Dunn 指数确定最优聚类数量。接着介绍参与者分配机制,并从数学上推导每个聚类中模型收敛所需通信轮数的表达式。此外,引入主从技术,通过知识蒸馏提升聚类中轻量级模型的性能。最后,开展实验评估以验证该方法的可行性和有效性,并将其与最先进技术进行比较。

英文摘要:

Federated Learning is a training framework that enables multiple participants to collaboratively train a shared model while preserving data privacy and minimizing communication overhead. The heterogeneity of devices and networking resources of the participants delay the training and aggregation in federated learning. This paper proposes a federated learning approach to manoeuvre the heterogeneity among the participants using resource aware clustering. The approach begins with the server gathering information about the devices and networking resources of participants, after which resource aware clustering is performed to determine the optimal number of clusters using Dunn Indices. The mechanism of participant assignment is then introduced, and the expression of communication rounds required for model convergence in each cluster is mathematically derived. Furthermore, a master-slave technique is introduced to improve the performance of the lightweight models in the clusters using knowledge distillation. Finally, experimental evaluations are conducted to verify the feasibility and effectiveness of the approach and to compare it with state-of-the-art techniques.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑