Federated Customization of Large Models: Approaches, Experiments, and Insights
College of Physics and Information Engineering, Fuzhou University（物理与信息工程学院，福州大学）；Data61, CSIRO（Data61，CSIRO）；Department of Computer Science and Information Technology, La Trobe University（计算机科学与信息科技系，拉筹夫大学）；College of Computing and Data Science, Nanyang Technological University（计算与数据科学学院，南洋理工大学）
Yuchuan Ye, Ming Ding, Youjia Chen, Peng Cheng, Dusit Niyato
In this article, we explore federated customization of large models and highlight the key challenges it poses within the federated learning framework. We review several popular large model customization techniques, including full fine-tuning, efficient fine-tuning, prompt engineering, prefix-tuning, knowledge distillation, and retrieval-augmented generation. Then, we discuss how these techniques can be implemented within the federated learning framework. Moreover, we conduct experiments on federated prefix-tuning, which, to the best of our knowledge, is the first trial to apply prefix-tuning in the federated learning setting. The conducted experiments validate its feasibility with performance close to centralized approaches. Further comparison with three other federated customization methods demonstrated its competitive performance, satisfactory efficiency, and consistent robustness.
Jiangsu University（江苏大学）；Southwest University（西南大学）；The University of Sydney（悉尼大学）；Monash University（莫纳什大学）；Sichuan University（四川大学）；Nanyang Technological University（南洋理工大学）
A Middle Path for On-Premises LLM Deployment: Preserving Privacy Without Sacrificing Model Confidentiality
Shanghai Jiao Tong University（上海交通大学）；National University of Defense Technology（国防科技大学）；Chinese University of Hong Kong (Shenzhen)（香港中文大学（深圳））；Tsinghua University（清华大学）
Navigating Text-To-Image Customization: From LyCORIS Fine-Tuning to Model Evaluation
University of Toronto（多伦多大学）；Yonsei University（延世大学）；University of Texas at San Antonio（德克萨斯大学圣安东尼奥分校）；National Tsing Hua University（国立清华大学）；Apple（苹果公司）
Label Granularity Skew in Federated Learning with Hierarchical Image Classification
School of Computer Science and Engineering, Soongsil University（崇实大学计算机科学与工程学院）；School of Computing and Augmented Intelligence, Arizona State University（亚利桑那州立大学计算与增强智能学院）
FedSDAF: Leveraging Source Domain Awareness for Enhanced Federated Domain Generalization
School of Computer Science and Technology, Huazhong University of Science and Technology（华中科技大学计算机科学与技术学院）；School of Software Engineering, Huazhong University of Science and Technology（华中科技大学软件工程学院）；School of Computer Science and Technology, Tiangong University（天津工业大学计算机科学与技术学院）
