arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2503.06962cs.LGcs.AI
收藏

捕获全局特征统计量以实现单轮联邦学习

Capture Global Feature Statistics for One-Shot Federated Learning

Institute of Information Engineering, Chinese Academy of Sciences(中国科学院信息工程研究所);School of Cyber Security, University of Chinese Academy of Sciences(中国科学院大学网络空间安全学院);State Key Laboratory of Cyberspace Security Defense(网络空间安全防御国家重点实验室)

Zenghao Guan, Yucan Zhou, Xiaoyan Gu

AI总结:

本文提出 FedCGS 算法,利用预训练模型捕获全局特征统计量,实现无需训练且抗数据异质性的单轮联邦学习,并扩展至个性化场景。

摘要:

传统联邦学习(FL)需要在服务器与客户端之间进行大量通信轮次,带来了通信成本高、连接中断风险大以及易受隐私攻击等显著挑战。单轮联邦学习(One-shot FL)通过仅需一轮通信即可训练全局服务器模型,已成为克服上述缺陷的一种极具吸引力的学习范式。然而,现有的单轮联邦学习方法在服务器或客户端上计算开销昂贵,且无法稳定有效地处理非独立同分布(non-IID)数据。为应对这些挑战,本文提出 FedCGS,一种利用预训练模型捕获全局特征统计量的新型联邦学习算法。借助全局特征统计量,我们实现了无需训练且对数据异质性具有鲁棒性的单轮联邦学习。此外,我们将其应用扩展到个性化场景,在该场景中客户端只需与服务器额外执行一轮通信以下载全局统计量。大量实验结果表明,我们的方法在多种数据异质性设置下均具有有效性。代码见 https://github.com/Yuqin-G/FedCGS。

英文摘要:

Traditional Federated Learning (FL) necessitates numerous rounds of communication between the server and clients, posing significant challenges including high communication costs, connection drop risks and susceptibility to privacy attacks. One-shot FL has become a compelling learning paradigm to overcome above drawbacks by enabling the training of a global server model via a single communication round. However, existing one-shot FL methods suffer from expensive computation cost on the server or clients and cannot deal with non-IID (Independent and Identically Distributed) data stably and effectively. To address these challenges, this paper proposes FedCGS, a novel Federated learning algorithm that Capture Global feature Statistics leveraging pre-trained models. With global feature statistics, we achieve training-free and heterogeneity-resistant one-shot FL. Furthermore, we extend its application to personalization scenario, where clients only need execute one extra communication round with server to download global statistics. Extensive experimental results demonstrate the effectiveness of our methods across diverse data heterogeneity settings. Code is available at https://github.com/Yuqin-G/FedCGS.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑