Federated Learning (FL) has become an established technique to facilitate privacy-preserving collaborative training across a multitude of clients. However, new approaches to FL often discuss their contributions involving small deep-learning models only and focus on training full models on clients. In the wake of Foundation Models (FM), the reality is different for many deep learning applications. Typically, FMs have already been pre-trained across a wide variety of tasks and can be fine-tuned to specific downstream tasks over significantly smaller datasets than required for full model training. However, access to such datasets is often challenging. By its design, FL can help to open data silos. With this survey, we introduce a novel taxonomy focused on computational and communication efficiency, the vital elements to make use of FMs in FL systems. We discuss the benefits and drawbacks of parameter-efficient fine-tuning (PEFT) for FL applications, elaborate on the readiness of FL frameworks to work with FMs, and provide future research opportunities on how to evaluate generative models in FL as well as the interplay of privacy and PEFT.
DP2FL: Dual Prompt Personalized Federated Learning in Foundation Models
College of Software, Jilin University(吉林大学软件学院);College of Computer Science and Technology, Jilin University(吉林大学计算机科学与技术学院);College of Computer Science and Engineering, Changchun University of Technology(长春理工大学计算机科学与工程学院)
A Survey of Personalized Federated Foundation Models for Privacy-Preserving Recommendation
Australian Artificial Intelligence Institute, University of Technology Sydney(澳大利亚人工智能研究所,悉尼技术大学);College of Computer Science and Technology, Jilin University(吉林大学计算机科学与技术学院);School of Computer Science and Technology, Beijing Jiaotong University(北京交通大学计算机科学与技术学院);Department of Data Science and Artificial Intelligence, The Hong Kong Polytechnic University(香港理工大学数据科学与人工智能系)
Synthetic Data Aided Federated Learning Using Foundation Models
The University of Manchester(曼彻斯特大学);Institute for Infocomm Research, A*STAR(新加坡资讯通信研究院(新加坡科技研究局下属机构));KU Leuven(鲁汶大学);COSIC, KU Leuven(鲁汶大学计算机与安全工业密码学中心)
FedMS: Federated Learning with Mixture of Sparsely Activated Foundations Models
The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳));Future Network of Intelligence Institute(未来智联网络研究院);Guangdong Provincial Key Laboratory of Future Networks of Intelligence(广东省未来智联网络重点实验室)
University of Technology Sydney(悉尼科技大学);Macquarie University(麦考瑞大学);University of New Castle(纽卡斯尔大学);University of Porto(波尔图大学);Nanyang Technological University(南洋理工大学);University of Minnesota(明尼苏达大学)
ZooPFL: Exploring Black-box Foundation Models for Personalized Federated Learning
Tsinghua University(清华大学);Microsoft Research(微软研究院);Idiap Research Institute(Idiap研究所);UIUC(伊利诺伊大学厄巴纳-香槟分校);Chinese Academy of Sciences(中国科学院);Hong Kong University of Science and Technology(香港科技大学)
Discrete Prototypical Memories for Federated Time Series Foundation Models
Australian Artificial Intelligence Institute, University of Technology Sydney(澳大利亚人工智能研究所,悉尼科技大学);The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));Shenzhen University(深圳大学);Institute of Computing Technology, Chinese Academy of Sciences(中国科学院计算技术研究所);Sichuan University(四川大学)
Synergizing Foundation Models and Federated Learning: A Survey
Uppsala University(乌普萨拉大学);University College London(伦敦大学学院);University of Liverpool(利物浦大学);Eindhoven University of Technology(埃因霍温理工大学);The University of Hong Kong(香港大学);Research Institutes of Sweden(瑞典研究机构)
Ten Challenging Problems in Federated Foundation Models
WeBank(微众银行);Universiti Malaya(马来亚大学);Institute of Computing Technology, Chinese Academy of Sciences(中国科学院计算技术研究所);Duke Kunshan University(昆山杜克大学);KTH Royal Institute of Technology(瑞典皇家理工学院);Beihang University(北京航空航天大学);Shanghai Jiao Tong University(上海交通大学);Xi’an Jiaotong University(西安交通大学);Southwestern University of Finance and Economic(西南财经大学)
Multi-Modal Multi-Task Federated Foundation Models for Next-Generation Extended Reality Systems: Towards Privacy-Preserving Distributed Intelligence in AR/VR/MR
University at Buffalo–SUNY(布法罗大学-纽约州立大学);Department of Electrical Engineering(电气工程系);New Jersey Institute of Technology (NJIT)(新泽西理工学院)