FedWon: 无需归一化实现多域联邦学习的成功
FedWon: Triumphing Multi-domain Federated Learning Without Normalization
AI总结：
针对多域联邦学习中特征分布差异导致的性能问题，提出移除归一化层、用缩放权重标准化重参数化卷积的FedWon方法，在多数据集和模型上准确率超现有方法最高10%以上，适配两类FL场景及资源受限设备，还能应对标签分布偏移。
摘要：
联邦学习（FL）通过在去中心化客户端上进行协作原位训练来增强数据隐私性。然而，FL 面临非独立同分布（non-i.i.d）数据带来的挑战，这会导致潜在的性能下降并阻碍收敛。现有研究主要解决标签分布偏移的问题，而我们的研究针对一个关键但常被忽视的问题——多域联邦学习。在该场景下，客户端的数据来自不同域，具有不同的特征分布而非标签分布。为解决联邦学习中的多域问题，我们提出了一种名为 Federated learning Without normalizations（FedWon，无归一化联邦学习）的新方法。FedWon 的灵感源于以下观察：批量归一化（BN）难以有效建模多域的统计信息，而现有归一化技术也各有局限。为解决这些问题，FedWon 移除了联邦学习中的归一化层，并使用缩放权重标准化对卷积层进行重参数化。我们在5个数据集和5个模型上开展了大量实验，综合实验结果表明，FedWon 在所有实验设置下均优于 FedAvg 和当前最优方法（FedBN），在部分域中实现了超过10%的显著准确率提升。此外，FedWon 适用于跨机构（cross-silo）和跨设备（cross-device）两种联邦学习场景，展现出强大的域泛化能力，即使在批量大小仅为1的情况下仍能保持优异性能，因此可适配资源受限的设备。另外，FedWon 还能有效应对标签分布偏移的挑战。
英文摘要：
Federated learning (FL) enhances data privacy with collaborative in-situ training on decentralized clients. Nevertheless, FL encounters challenges due to non-independent and identically distributed (non-i.i.d) data, leading to potential performance degradation and hindered convergence. While prior studies predominantly addressed the issue of skewed label distribution, our research addresses a crucial yet frequently overlooked problem known as multi-domain FL. In this scenario, clients' data originate from diverse domains with distinct feature distributions, instead of label distributions. To address the multi-domain problem in FL, we propose a novel method called Federated learning Without normalizations (FedWon). FedWon draws inspiration from the observation that batch normalization (BN) faces challenges in effectively modeling the statistics of multiple domains, while existing normalization techniques possess their own limitations. In order to address these issues, FedWon eliminates the normalization layers in FL and reparameterizes convolution layers with scaled weight standardization. Through extensive experimentation on five datasets and five models, our comprehensive experimental results demonstrate that FedWon surpasses both FedAvg and the current state-of-the-art method (FedBN) across all experimental setups, achieving notable accuracy improvements of more than 10% in certain domains. Furthermore, FedWon is versatile for both cross-silo and cross-device FL, exhibiting robust domain generalization capability, showcasing strong performance even with a batch size as small as 1, thereby catering to resource-constrained devices. Additionally, FedWon can also effectively tackle the challenge of skewed label distribution.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
