SphereFed:超球面联邦学习
SphereFed: Hyperspherical Federated Learning
摘要:
联邦学习旨在从多个分散的设备(即客户端)训练全局模型,而无需交换其私有本地数据。一个关键的挑战是如何处理非独立身份证。(独立同分布的)跨多个客户端的数据,这可能引起它们的局部特征的差异。我们引入超球面联邦学习(SphereFed)框架来解决非独立同分布问题。通过将学习到的数据点的表示约束在由客户端共享的单位超球体上来发布。具体来说,所有客户端通过最小化相对于其权重跨越单位超球体的固定分类器的损失来学习其本地表示。在改进全局模型的联邦训练之后,通过最小化均方损失,用封闭形式的解进一步校准该分类器。我们表明,校准解决方案可以有效地计算和分布式,而无需直接存取的本地数据。大量的实验表明,我们的SphereFed方法能够大幅提高多个现有联邦学习算法的准确性(在具有挑战性的数据集上高达6%),并提高数据集和模型架构之间的计算和通信效率。
英文摘要:
Federated Learning aims at training a global model from multiple decentralized devices (i.e. clients) without exchanging their private local data. A key challenge is the handling of non-i.i.d. (independent identically distributed) data across multiple clients that may induce disparities of their local features. We introduce the Hyperspherical Federated Learning (SphereFed) framework to address the non-i.i.d. issue by constraining learned representations of data points to be on a unit hypersphere shared by clients. Specifically, all clients learn their local representations by minimizing the loss with respect to a fixed classifier whose weights span the unit hypersphere. After federated training in improving the global model, this classifier is further calibrated with a closed-form solution by minimizing a mean squared loss. We show that the calibration solution can be computed efficiently and distributedly without direct access of local data. Extensive experiments indicate that our SphereFed approach is able to improve the accuracy of multiple existing federated learning algorithms by a considerable margin (up to 6% on challenging datasets) with enhanced computation and communication efficiency across datasets and model architectures.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。