arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2211.05554cs.LGcs.CVcs.DC
收藏

基于聚合优化的抗数据异构和中毒攻击的鲁棒联邦学习

Robust Federated Learning against both Data Heterogeneity and Poisoning Attack via Aggregation Optimization

The Hong Kong University of Science and Technology(香港科技大学);Microsoft Research Asia(微软亚洲研究院)

Yueqi Xie, Weizhong Zhang, Renjie Pi, Fangzhao Wu, Qifeng Chen, Xing Xie, Sunghun Kim

摘要:

跨客户端的非IID数据分发和中毒攻击是现实世界的联邦学习(FL)系统中的两个主要挑战。虽然他们都吸引了很大的研究兴趣与特定的策略开发,没有已知的解决方案管理,以解决他们在一个统一的框架。为了普遍克服这两个挑战,我们提出了SmartFL,一个通用的方法,优化服务器端的聚合过程与少量的代理数据收集的服务提供商本身通过子空间训练技术。具体地,使用服务器收集的代理数据来优化每个参与客户端在每轮的聚合权重,这实质上是客户端模型所跨越的凸包中的全局模型的优化。由于在每一轮,在服务器端优化的可调参数的数量等于参与客户端的数量(因此与模型大小无关),因此我们能够仅使用少量代理数据(例如,约100个样品)。通过优化的聚合,SmartFL确保了对异构和恶意客户端的鲁棒性,这在现实世界的FL中是理想的,其中一个或两个问题都可能发生。我们提供的SmartFL的收敛性和泛化能力的理论分析。根据经验,SmartFL在使用非IID数据分发的FL和使用恶意客户端的FL上都实现了最先进的性能。源代码将被释放。

英文摘要:

Non-IID data distribution across clients and poisoning attacks are two main challenges in real-world federated learning (FL) systems. While both of them have attracted great research interest with specific strategies developed, no known solution manages to address them in a unified framework. To universally overcome both challenges, we propose SmartFL, a generic approach that optimizes the server-side aggregation process with a small amount of proxy data collected by the service provider itself via a subspace training technique. Specifically, the aggregation weight of each participating client at each round is optimized using the server-collected proxy data, which is essentially the optimization of the global model in the convex hull spanned by client models. Since at each round, the number of tunable parameters optimized on the server side equals the number of participating clients (thus independent of the model size), we are able to train a global model with massive parameters using only a small amount of proxy data (e.g., around one hundred samples). With optimized aggregation, SmartFL ensures robustness against both heterogeneous and malicious clients, which is desirable in real-world FL where either or both problems may occur. We provide theoretical analyses of the convergence and generalization capacity for SmartFL. Empirically, SmartFL achieves state-of-the-art performance on both FL with non-IID data distribution and FL with malicious clients. The source code will be released.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑