arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2410.22192cs.LGcs.ITeess.SPmath.ITstat.ML
收藏

$r$Age-$k$:使用年龄因子的通信高效联邦学习

$r$Age-$k$: Communication-Efficient Federated Learning Using Age Factor

University of Maryland(马里兰大学)

Matin Mortaheb, Priyanka Kaswan, Sennur Ulukus

AI总结:

针对联邦学习的数据异构性与通信开销问题,提出基于信息年龄的rAge-k算法,通过年龄向量选择性请求更新与客户端聚类,在非独立同分布数据集上实现了更优的训练效率。

摘要:

联邦学习(FL)是一种由参数服务器(PS)协调多个客户端共同训练统一机器学习模型的协作方法。然而,该方法面临两大核心挑战:数据异构性与通信开销。数据异构性指不同客户端的异构数据导致模型训练出现不一致的问题;通信开销则源于参数服务器与客户端之间交换的大量参数更新。现有解决方案通常分别应对这两类挑战。本文提出了一种新型通信高效算法,该算法利用信息年龄指标同时解决联邦学习的上述两大局限。我们在参数服务器端引入年龄向量,用于追踪不同模型参数从客户端接收更新的频率。参数服务器利用该向量有选择地向每个客户端请求特定梯度索引的更新。此外,参数服务器借助年龄向量识别数据统计特征相似的客户端,并将其划分为集群,再结合集群内客户端的年龄向量,高效协调集群内客户端间的梯度索引更新。我们在高度非独立同分布(non-i.i.d.)设置下,使用MNIST和CIFAR10数据集对所提方法进行了评估。实验结果表明,该方法能够加快训练速度,在效率上优于其他通信高效策略。

英文摘要:

Federated learning (FL) is a collaborative approach where multiple clients, coordinated by a parameter server (PS), train a unified machine-learning model. The approach, however, suffers from two key challenges: data heterogeneity and communication overhead. Data heterogeneity refers to inconsistencies in model training arising from heterogeneous data at different clients. Communication overhead arises from the large volumes of parameter updates exchanged between the PS and clients. Existing solutions typically address these challenges separately. This paper introduces a new communication-efficient algorithm that uses the age of information metric to simultaneously tackle both limitations of FL. We introduce age vectors at the PS, which keep track of how often the different model parameters are updated from the clients. The PS uses this to selectively request updates for specific gradient indices from each client. Further, the PS employs age vectors to identify clients with statistically similar data and group them into clusters. The PS combines the age vectors of the clustered clients to efficiently coordinate gradient index updates among clients within a cluster. We evaluate our approach using the MNIST and CIFAR10 datasets in highly non-i.i.d. settings. The experimental results show that our proposed method can expedite training, surpassing other communication-efficient strategies in efficiency.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑