arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2410.01502cs.DC
收藏

有限存储下数据动态异构性的个性化联邦学习

Personalized Federated Learning on Data with Dynamic Heterogeneity under Limited Storage

Sixing Tan, Xianmin Liu

AI总结:

针对数据动态异构与存储受限导致的局部灾难性遗忘及协作度难估计问题,提出基于低消耗生成式回放架构的个性化联邦学习框架pFedGRP,通过按类别解耦生成器、局部模型优化及数据重构实现个性化聚合与知识转移,在五个数据集上优于八种基线方法。

摘要:

近期,信息化开放的大量数据源加剧了数据异构性,数据生成速度加快及数据法规的逐步实施限制了数据的存储时间。在个性化联邦学习(pFL)中,客户端训练定制化模型以满足其个人目标。然而,由于时变的局部数据异构性和先前数据的不可访问性,现有的pFL方法不仅无法解决局部模型的灾难性遗忘,也难以估计客户端间的协作程度。为解决此问题,我们的核心思想是构建一种低消耗、高质量的生成式回放架构。具体而言,我们按类别解耦生成器以减少各类别的生成误差并缓解灾难性遗忘,利用局部模型提升生成数据质量并降低生成器的更新频率,并提出局部数据重构方案以减少数据生成同时调整数据类别比例。基于上述内容,我们提出了pFL框架pFedGRP,以实现个性化聚合与局部知识转移。在五个数据集上的多项设置综合实验表明,pFedGRP优于八种基线方法。

英文摘要:

Recently, a large number of data sources opened up by informatization intensify the data heterogeneity, the faster speed of data generation and the gradual implementation of data regulations limit the storage time of data. In personalized Federated Learning (pFL), clients train customized models to meet their personal objectives. However, due to the time-varying local data heterogeneity and the inaccessibility of previous data, existing pFL methods not only fail to solve the catastrophic forgetting of local models, but also difficult to estimate the degree of collaboration between clients. To address this issue, our core idea is a low consumption and high-quality generative replay architecture. Specifically, we decouple the generator by category to reduce the generation error of each category while mitigating catastrophic forgetting, use local model to improving the quality of generated data and reducing the update frequency of generator, and propose a local data reconstruction scheme to reduce data generation while adjusting the proportion of data categories. Based on above, we propose our pFL framework, pFedGRP, to achieve personalized aggregation and local knowledge transfer. Comprehensive experiments on five datasets with multiple settings show the superiority of pFedGRP over eight baseline methods.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑