arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2405.06993cs.LGcs.DC
收藏

面向异构联邦学习的鲁棒模型聚合:分析与优化

Robust Model Aggregation for Heterogeneous Federated Learning: Analysis and Optimizations

School of Electronic and Optical Engineering, Nanjing University of Science and Technology(南京理工大学电子工程与光电技术学院);Department of Computing, The Hong Kong Polytechnic University(香港理工大学计算系);Data61, CSIRO(联邦科学与工业研究组织数据61分部);Digital Economy Research Institute, Nanjing University of Science and Technology(南京理工大学数字经济研究院);Institute of High Performance Computing, A*STAR(新加坡科技研究局高性能计算研究所);Digital Research Institute, ENN Group(新奥集团数字研究院);Department of Electronic Engineering, Shanghai Jiao Tong University(上海交通大学电子工程系);National Mobile Communications Research Laboratory, Southeast University(东南大学移动通信国家重点实验室)

Yumeng Shao, Jun Li, Long Shi, Kang Wei, Ming Ding, Qianmu Li, Zengxiang Li, Wen Chen, Shi Jin

AI总结:

针对异构联邦学习中传统同步框架性能下降、异步框架存在更新不一致的问题,提出时间驱动同步联邦学习框架与判别式模型选择算法,可降低延迟并提升准确率。

摘要:

传统同步联邦学习(SFL)框架在异构系统中会因客户端本地数据规模不均衡、计算能力存在差异而出现性能下降。为解决该问题,异步联邦学习(AFL)和半异步联邦学习被提出,通过允许异步聚合来弥补性能损失。但异步聚合会引发新的问题:本地更新与全局更新之间存在不一致。受传统SFL和AFL现存问题的启发,我们首先提出了一种适用于异构系统的时间驱动同步联邦学习(T-SFL)框架。T-SFL的核心思想是服务器以固定时间间隔聚合来自不同客户端的模型,这些客户端各自完成的迭代次数各不相同。为评估T-SFL的学习性能,我们推导了全局损失函数的上界。进一步地,我们通过优化聚合权重来最小化所推导的上界。随后,我们提出了判别式模型选择(DMS)算法,该算法会剔除迭代次数低于预设阈值的客户端的本地模型。具体而言,该算法能够确保每个客户端的聚合权重准确反映其对全局模型更新的真实贡献,从而提升系统的效率与鲁棒性。为验证搭载DMS算法的T-SFL的有效性,我们使用MNIST、Cifar-10、Fashion-MNIST和SVHN等多个常用数据集开展了大量实验。实验结果表明,搭载DMS算法的T-SFL可将传统SFL的延迟降低50%,同时相较于当前最优的AFL算法,学习准确率平均提升3%。

英文摘要:

Conventional synchronous federated learning (SFL) frameworks suffer from performance degradation in heterogeneous systems due to imbalanced local data size and diverse computing power on the client side. To address this problem, asynchronous FL (AFL) and semi-asynchronous FL have been proposed to recover the performance loss by allowing asynchronous aggregation. However, asynchronous aggregation incurs a new problem of inconsistency between local updates and global updates. Motivated by the issues of conventional SFL and AFL, we first propose a time-driven SFL (T-SFL) framework for heterogeneous systems. The core idea of T-SFL is that the server aggregates the models from different clients, each with varying numbers of iterations, at regular time intervals. To evaluate the learning performance of T-SFL, we provide an upper bound on the global loss function. Further, we optimize the aggregation weights to minimize the developed upper bound. Then, we develop a discriminative model selection (DMS) algorithm that removes local models from clients whose number of iterations falls below a predetermined threshold. In particular, this algorithm ensures that each client's aggregation weight accurately reflects its true contribution to the global model update, thereby improving the efficiency and robustness of the system. To validate the effectiveness of T-SFL with the DMS algorithm, we conduct extensive experiments using several popular datasets including MNIST, Cifar-10, Fashion-MNIST, and SVHN. The experimental results demonstrate that T-SFL with the DMS algorithm can reduce the latency of conventional SFL by 50\%, while achieving an average 3\% improvement in learning accuracy over state-of-the-art AFL algorithms.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑