Capture Global Feature Statistics for One-Shot Federated Learning
Institute of Information Engineering, Chinese Academy of Sciences(中国科学院信息工程研究所);School of Cyber Security, University of Chinese Academy of Sciences(中国科学院大学网络空间安全学院);State Key Laboratory of Cyberspace Security Defense(网络空间安全防御国家重点实验室)
Traditional Federated Learning (FL) necessitates numerous rounds of communication between the server and clients, posing significant challenges including high communication costs, connection drop risks and susceptibility to privacy attacks. One-shot FL has become a compelling learning paradigm to overcome above drawbacks by enabling the training of a global server model via a single communication round. However, existing one-shot FL methods suffer from expensive computation cost on the server or clients and cannot deal with non-IID (Independent and Identically Distributed) data stably and effectively. To address these challenges, this paper proposes FedCGS, a novel Federated learning algorithm that Capture Global feature Statistics leveraging pre-trained models. With global feature statistics, we achieve training-free and heterogeneity-resistant one-shot FL. Furthermore, we extend its application to personalization scenario, where clients only need execute one extra communication round with server to download global statistics. Extensive experimental results demonstrate the effectiveness of our methods across diverse data heterogeneity settings. Code is available at https://github.com/Yuqin-G/FedCGS.
FedLPA: One-shot Federated Learning with Layer-Wise Posterior Aggregation
National University of Singapore(新加坡国立大学);Northeastern University(东北大学);University of Technology Sydney(悉尼科技大学);Tsinghua University(清华大学);ETH Zurich(苏黎世联邦理工学院)
The Gaussian-Head OFL Family: One-Shot Federated Learning from Client Global Statistics
Department of Sciences and Methods for Engineering(工程科学与方法系);Artificial Intelligence Research and Innovation Center(人工智能研究与创新中心);University of Modena and Reggio Emilia(摩德纳和雷吉奥艾米利亚大学)
One-shot Federated Learning Methods: A Practical Guide
National University of Singapore(新加坡国立大学);The Hong Kong University of Science and Technology(香港科技大学);ETH Zürich(苏黎世联邦理工学院);Zhejiang University of Finance & Economics Dongfang College(浙江财经大学东方学院);California Institute of Technology(加州理工学院);College of Computer Science and Technology, Zhejiang University(浙江大学计算机科学与技术学院);Hong Kong Baptist University(香港浸会大学)
Towards One-shot Federated Learning: Advances, Challenges, and Future Directions
Department of Electrical Engineering and Information Technology, University of Naples Federico II(那不勒斯费德里科二世大学电气工程与信息科技系);Department of Mathematics and Applications "R. Caccioppoli", University of Naples Federico II(那不勒斯费德里科二世大学数学与应用系);Department of Mathematics, Indian Institute of Technology Indore(印度理工学院Indore分校数学系)
Task Arithmetic Through The Lens Of One-Shot Federated Learning
Princeton University(普林斯顿大学);Fujitsu Research of America(富士通美国研究部);Operations Research and Financial Engineering(运筹学与金融工程);Electrical and Computer Engineering(电气与计算机工程)
Adaptive Federated Few-Shot Rare-Disease Diagnosis with Energy-Aware Secure Aggregation
School of Information, Computer, and Communication Technology(信息、计算机与通信技术学院);Sirindhorn International Institute of Technology, Thammasat University(泰国朱拉隆功大学国际技术学院)
When to Stop Federated Learning: Zero-Shot Generation of Synthetic Validation Data with Generative AI for Early Stopping
Institute of Information & Communications Technology Planning & Evaluation (IITP)-ITRC (Information Technology Research Center)(信息与通信技术规划与评估机构(IITP)-ITRC(信息技术研究中心));Institute of Science Tokyo(东京科学研究所)
Towards a Larger Model via One-Shot Federated Learning on Heterogeneous Client Models
Advanced Wireless Technology Laboratory, Munich Research Center, Huawei Technologies Duesseldorf GmbH(慕尼黑研究中心先进无线技术实验室,华为技术有限公司);TUM School of Computation, Information and Technology, Technical University of Munich(慕尼黑技术大学计算、信息与技术学院)
XOR Mixup: Privacy-Preserving Data Augmentation for One-Shot Federated Learning
Seoul National University Hospital(首尔国立大学医院);Chung-Ang University(中央大学);Korea University(高丽大学);Deakin University(迪肯大学);University of Oulu(奥卢大学);Yonsei University(延世大学)
Zero-Shot Federated Learning with New Classes for Audio Classification
Global AI Accelerator, Ericsson(爱立信全球人工智能加速器);Ericsson Research(爱立信研究院)
联邦学习是一种从不同用户设备中提取见解同时保护用户隐私的有效方法。然而,具有完全不可见的数据分布的新类可以在联邦学习设置中的任何设备上流式传输,其数据不能被全局服务器或其他用户访问。为此,我们提出了一个统一的zero-shot框架来处理联邦学习过程中的这些挑战。我们在这里模拟了两种情况- 1)当用户没有报告新的类别标签时,使用传统的FL设置; 2)当用户报告新的类别标签时,我们通过计算与每个设备的新类别对应的类别相似性矩阵来合成Ancesterized Data Imperial,然后进行无监督聚类以区分不同用户的新类别。此外,我们提出的框架还可以处理参与用户的标签和模型中的统计异质性。我们在两个广泛使用的音频分类应用程序(关键字识别和城市声音分类)上,通过本地和全局更新中的新类别以及异构标签和模型,在不同的通信回合(FL迭代)中对我们的设备上框架进行了经验评估,并观察到平均确定性准确度分别增加了~4.041%和~4.258%。
FedBiCross: Personalized One-Shot Federated Learning on Medical Images
School of Computer Science and Engineering, Northwestern Polytechnical University, China(西北工业大学计算机科学与工程学院);School of Science and Technology, Hong Kong Metropolitan University, Hong Kong(香港 Metropolitan 大学科学与技术学院);Department of Computer Science, Hong Kong Baptist University, Hong Kong(香港 Baptist 大学计算机科学系)
A Unified Solution to Diverse Heterogeneities in One-shot Federated Learning
McGill University(麦吉尔大学);Mila-Quebec AI Institute(魁北克AI研究院);The University of Adelaide(阿德莱德大学);University of Southern Queensland(南方昆士兰大学);Deakin University(德肯大学);Monash University(墨尔本大学)
Enhancing One-Shot Federated Learning Through Data and Ensemble Co-Boosting
University of Science and Technology of China(中国科学技术大学);Hong Kong Baptist University(香港浸会大学);University of Maryland College Park(马里兰大学学院公园分校);The University of Sydney(悉尼大学)