arXivDaily arXiv每日学术速递 周一至周五更新
​通知:联邦节假日调休:哥伦布日 / 原住民日(Columbus Day / Indigenous Peoples' Day),今日(13日)无更新。
arXiv 2501.11002cs.LGcs.DC

pMixFed:通过自适应逐层 Mixup 实现高效的个性化联邦学习

pMixFed: Efficient Personalized Federated Learning through Adaptive Layer-Wise Mixup

Florida International University(佛罗里达国际大学);USC Information Sciences Institute(南加州大学信息科学研究所)

Yasaman Saadati, Mohammad Rostami, M. Hadi Amini

AI总结:

pMixFed 提出一种动态逐层个性化联邦学习方法,通过在全局与局部模型间引入自适应 mixup、渐进个性化过渡和新聚合机制,缓解客户端漂移与灾难性遗忘,提升非IID场景下的精度与训练效率。

摘要:

传统联邦学习(FL)方法在处理异构数据以及为非独立同分布(non-IID)场景提供个性化解决方案时面临显著挑战。个性化联邦学习(PFL)方法旨在通过平衡泛化与个性化来解决这些问题,通常采用参数解耦或部分模型的方式,冻结部分神经网络层用于个性化,同时对其余层进行全局聚合。然而,现有方法仍然面临全局-局部模型不一致、客户端漂移和灾难性遗忘等挑战,这些问题会降低模型精度。为克服这些局限,我们提出了 pMixFed,这是一种动态的、逐层的 PFL 方法,在共享的全局模型与个性化局部模型之间引入 mixup。我们的方法引入了一种自适应策略,用于划分个性化层与共享层;一种个性化程度的渐进过渡机制,以增强本地客户端适应能力;改进的跨客户端泛化能力;以及一种新颖的聚合机制,以缓解灾难性遗忘。大量实验表明,pMixFed 优于最先进的 PFL 方法,在不同异构设置下展现出更快的模型训练速度、更强的鲁棒性以及更好的数据异构处理能力。

英文摘要:

Traditional Federated Learning (FL) methods encounter significant challenges when dealing with heterogeneous data and providing personalized solutions for non-IID scenarios. Personalized Federated Learning (PFL) approaches aim to address these issues by balancing generalization and personalization, often through parameter decoupling or partial models that freeze some neural network layers for personalization while aggregating other layers globally. However, existing methods still face challenges of global-local model discrepancy, client drift, and catastrophic forgetting, which degrade model accuracy. To overcome these limitations, we propose $\textit{pMixFed}$, a dynamic, layer-wise PFL approach that integrates $\textit{mixup}$ between shared global and personalized local models. Our method introduces an adaptive strategy for partitioning between personalized and shared layers, a gradual transition of personalization degree to enhance local client adaptation, improved generalization across clients, and a novel aggregation mechanism to mitigate catastrophic forgetting. Extensive experiments demonstrate that pMixFed outperforms state-of-the-art PFL methods, showing faster model training, increased robustness, and improved handling of data heterogeneity under different heterogeneous settings.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑