FedBAT：通过可学习二值化实现通信高效的联邦学习
FedBAT: Communication-Efficient Federated Learning via Learnable Binarization
AI总结：
FedBAT 提出在联邦学习本地训练中直接学习二值模型更新，通过可学习二值化算子减少近似误差并降低通信开销，实验表明其收敛更快且精度最高可超基线 9%。
摘要：
联邦学习是一种很有前景的分布式机器学习范式，能够在不暴露用户隐私的情况下有效利用大规模数据。然而，它可能产生显著的通信开销，从而可能损害训练效率。为应对这一挑战，大量研究建议对模型更新进行二值化。尽管如此，传统方法通常以训练后处理的方式对模型更新进行二值化，导致显著的近似误差以及随之而来的模型精度下降。为此，我们提出联邦二值化感知训练（FedBAT），这是一个新颖的框架，在本地训练过程中直接学习二值模型更新，从而从本质上减少近似误差。FedBAT 引入了一种创新的二值化算子，并精心设计了导数以促进高效学习。此外，我们建立了关于 FedBAT 收敛性的理论保证。在四个常用数据集上进行了大量实验。结果表明，FedBAT 显著加速了收敛，并且精度超过基线最多达 9%，在某些情况下甚至超过了 FedAvg。
英文摘要：
Federated learning is a promising distributed machine learning paradigm that can effectively exploit large-scale data without exposing users' privacy. However, it may incur significant communication overhead, thereby potentially impairing the training efficiency. To address this challenge, numerous studies suggest binarizing the model updates. Nonetheless, traditional methods usually binarize model updates in a post-training manner, resulting in significant approximation errors and consequent degradation in model accuracy. To this end, we propose Federated Binarization-Aware Training (FedBAT), a novel framework that directly learns binary model updates during the local training process, thus inherently reducing the approximation errors. FedBAT incorporates an innovative binarization operator, along with meticulously designed derivatives to facilitate efficient learning. In addition, we establish theoretical guarantees regarding the convergence of FedBAT. Extensive experiments are conducted on four popular datasets. The results show that FedBAT significantly accelerates the convergence and exceeds the accuracy of baselines by up to 9\%, even surpassing that of FedAvg in some cases.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
