arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2205.03993cs.DC
收藏

面向个性化联邦学习的分层模型聚合

Layer-wised Model Aggregation for Personalized Federated Learning

Xiaosong Ma, Jie Zhang, Song Guo, Wenchao Xu

摘要:

个性化联邦学习(Personalized Federated Learning,pFL)不仅可以从广泛的分布式数据中获取共同的先验知识,而且可以为异构客户提供定制的模型。过去几年的研究都采用加权聚合的方式来生成个性化模型,其中权重是通过校准整个模型参数或损失值的距离来确定的,并且尚未考虑层级对聚合过程的影响,导致模型收敛滞后,并且在非IID数据集上的个性化不足。在本文中,我们提出了一种新的pFL训练框架,称为层明智的个性化联邦学习(pFedLA),可以辨别每个层的重要性,从不同的客户端,从而能够优化客户端的异构数据的个性化模型聚合。具体来说,我们在服务器端为每个客户端使用一个专用的超网络,该超网络经过训练以识别层粒度的相互贡献因素。同时,引入参数化机制来更新逐层聚合权重,以逐步利用用户间的相似性,实现精确的模型个性化。在不同的模型和学习任务进行了广泛的实验,我们表明,所提出的方法实现了显着更高的性能比国家的最先进的pFL方法。

英文摘要:

Personalized Federated Learning (pFL) not only can capture the common priors from broad range of distributed data, but also support customized models for heterogeneous clients. Researches over the past few years have applied the weighted aggregation manner to produce personalized models, where the weights are determined by calibrating the distance of the entire model parameters or loss values, and have yet to consider the layer-level impacts to the aggregation process, leading to lagged model convergence and inadequate personalization over non-IID datasets. In this paper, we propose a novel pFL training framework dubbed Layer-wised Personalized Federated learning (pFedLA) that can discern the importance of each layer from different clients, and thus is able to optimize the personalized model aggregation for clients with heterogeneous data. Specifically, we employ a dedicated hypernetwork per client on the server side, which is trained to identify the mutual contribution factors at layer granularity. Meanwhile, a parameterized mechanism is introduced to update the layer-wised aggregation weights to progressively exploit the inter-user similarity and realize accurate model personalization. Extensive experiments are conducted over different models and learning tasks, and we show that the proposed methods achieve significantly higher performance than state-of-the-art pFL methods.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑