使用顺序训练的超级客户端加速异构联邦学习
Speeding up Heterogeneous Federated Learning with Sequentially Trained Superclients
摘要：
联邦学习（FL）允许在隐私受限的场景中训练机器学习模型，实现边缘设备的协作，而无需本地数据共享。由于本地数据集的不同统计分布和客户端的计算异构性，这种方法提出了一些挑战。特别是，高度非i.i.d.数据严重地损害了训练的神经网络的性能及其收敛速率，增加了为达到与集中式方案的性能相当的性能所要求的通信循环的数量。作为一种解决方案，我们提出了FedSeq，一种新的框架，利用异构客户端，即超级客户端的子组的顺序训练，以隐私兼容的方式模拟集中式范式。给定一个固定的预算的通信轮，我们表明，FedSeq优于或匹配几个国家的最先进的联邦算法的最终性能和收敛速度。最后，我们的方法可以很容易地与文献中的其他方法相结合。实验结果表明，将现有算法与FedSeq相结合，进一步提高了其最终性能和收敛速度。我们在CIFAR-10和CIFAR-100上测试了我们的方法，并证明了它在i.i.d.和非I.I.D.场景
英文摘要：
Federated Learning (FL) allows training machine learning models in privacy-constrained scenarios by enabling the cooperation of edge devices without requiring local data sharing. This approach raises several challenges due to the different statistical distribution of the local datasets and the clients' computational heterogeneity. In particular, the presence of highly non-i.i.d. data severely impairs both the performance of the trained neural network and its convergence rate, increasing the number of communication rounds requested to reach a performance comparable to that of the centralized scenario. As a solution, we propose FedSeq, a novel framework leveraging the sequential training of subgroups of heterogeneous clients, i.e. superclients, to emulate the centralized paradigm in a privacy-compliant way. Given a fixed budget of communication rounds, we show that FedSeq outperforms or match several state-of-the-art federated algorithms in terms of final performance and speed of convergence. Finally, our method can be easily integrated with other approaches available in the literature. Empirical results show that combining existing algorithms with FedSeq further improves its final performance and convergence speed. We test our method on CIFAR-10 and CIFAR-100 and prove its effectiveness in both i.i.d. and non-i.i.d. scenarios.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
