arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2012.08336cs.LGcs.DCcs.NImath.OC
收藏

经济高效的联邦学习设计

Cost-Effective Federated Learning Design

Shenzhen Institute of Artificial Intelligence and Robotics for Society(深圳市人工智能与机器人研究院);The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳));IBM T. J. Watson Research Center(IBM托马斯·J·沃森研究中心);Yale University(耶鲁大学)

Bing Luo, Xiang Li, Shiqiang Wang, Jianwei Huang, Leandros Tassiulas

摘要:

联邦学习(FL)是一种分布式学习范式,它使大量设备能够协作学习模型,而无需共享它们的原始数据。尽管其实际效率和有效性,但迭代的设备上学习过程在学习时间和能耗方面产生了相当大的成本,这在很大程度上取决于所选择的客户端的数量和每个训练轮中的本地迭代次数。在本文中,我们分析了如何设计自适应FL,最佳地选择这些基本的控制变量,以最小化总成本,同时确保收敛。在理论上,我们解析地建立了总成本与控制变量之间的关系,并给出了收敛上界。为了有效地解决成本最小化问题,我们开发了一个低成本的基于采样的算法来学习收敛相关的未知参数。我们得到重要的解决方案的属性,有效地识别不同的度量偏好的设计原则。实际上,我们评估我们的理论结果在模拟环境和硬件原型。实验证据验证了我们推导的属性,并表明我们提出的解决方案实现了接近最佳的性能,为各种数据集,不同的机器学习模型,异构系统设置。

英文摘要:

Federated learning (FL) is a distributed learning paradigm that enables a large number of devices to collaboratively learn a model without sharing their raw data. Despite its practical efficiency and effectiveness, the iterative on-device learning process incurs a considerable cost in terms of learning time and energy consumption, which depends crucially on the number of selected clients and the number of local iterations in each training round. In this paper, we analyze how to design adaptive FL that optimally chooses these essential control variables to minimize the total cost while ensuring convergence. Theoretically, we analytically establish the relationship between the total cost and the control variables with the convergence upper bound. To efficiently solve the cost minimization problem, we develop a low-cost sampling-based algorithm to learn the convergence related unknown parameters. We derive important solution properties that effectively identify the design principles for different metric preferences. Practically, we evaluate our theoretical results both in a simulated environment and on a hardware prototype. Experimental evidence verifies our derived properties and demonstrates that our proposed solution achieves near-optimal performance for various datasets, different machine learning models, and heterogeneous system settings.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑