用于按需和原位定制的高效拆分混合联邦学习
Efficient Split-Mix Federated Learning for On-Demand and In-Situ Customization
摘要：
联邦学习（FL）为多个参与者提供了一个分布式的学习框架，使他们可以在不共享原始数据的情况下进行协作学习。在许多实际FL场景中，由于硬件和推理动态的差异，参与者具有异构资源，需要快速加载不同大小和鲁棒性级别的模型。异质性和动态性共同对现有的FL方法提出了重大挑战，从而极大地限制了FL的适用性。在本文中，我们提出了一种新的分裂混合FL策略的异构参与者，一旦训练完成，提供原位定制的模型大小和鲁棒性。具体来说，我们通过学习一组不同大小和鲁棒性水平的基础子网络来实现定制，然后根据推理要求按需聚合这些子网络。这种拆分混合策略实现了定制，在通信，存储和推理方面具有高效率。大量的实验表明，我们的方法提供了更好的原位定制比现有的异构架构FL方法。代码和预训练模型可在https://github.com/illidanlab/SplitMix上找到。
英文摘要：
Federated learning (FL) provides a distributed learning framework for multiple participants to collaborate learning without sharing raw data. In many practical FL scenarios, participants have heterogeneous resources due to disparities in hardware and inference dynamics that require quickly loading models of different sizes and levels of robustness. The heterogeneity and dynamics together impose significant challenges to existing FL approaches and thus greatly limit FL's applicability. In this paper, we propose a novel Split-Mix FL strategy for heterogeneous participants that, once training is done, provides in-situ customization of model sizes and robustness. Specifically, we achieve customization by learning a set of base sub-networks of different sizes and robustness levels, which are later aggregated on-demand according to inference requirements. This split-mix strategy achieves customization with high efficiency in communication, storage, and inference. Extensive experiments demonstrate that our method provides better in-situ customization than the existing heterogeneous-architecture FL methods. Codes and pre-trained models are available: https://github.com/illidanlab/SplitMix.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
