arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2406.14910cs.LGcs.DCmath.OC
收藏

面向分层联邦学习中的动态资源分配与客户端调度:一种两阶段深度强化学习方法

Towards Dynamic Resource Allocation and Client Scheduling in Hierarchical Federated Learning: A Two-Phase Deep Reinforcement Learning Approach

Shanghai University(上海大学);Commonwealth Scientific and Industrial Research Organization (CSIRO)(英联邦科学与工业研究组织);Fudan University(复旦大学);Xidian University(西安电子科技大学)

Xiaojing Chen, Zhenyuan Li, Wei Ni, Xin Wang, Shunqing Zhang, Yanzan Sun, Shugong Xu, Qingqi Pei

AI总结:

针对能量收集驱动的分层联邦学习系统的多级资源与客户端调度问题,提出两阶段DDPG框架TP-DDPG,结合SCABA算法平衡学习延迟与模型精度,在参数更少的情况下可将训练时间缩短39.4%。

摘要:

联邦学习(FL)是一种无需共享数据即可训练共享机器学习模型的可行技术。分层联邦学习(HFL)系统在其多级能量、计算、通信及客户端调度方面仍有待研究,尤其是针对依赖能量收集为自身运行供电的客户端的场景。本文提出了一种全新的两阶段深度确定性策略梯度(DDPG)框架,命名为“TP-DDPG”,用于在能量收集驱动的HFL系统中在线平衡FL过程的学习延迟与模型精度。其核心思路是将优化决策分为两组,第一阶段采用DDPG学习其中一组决策,同时将另一组决策视为环境的一部分,为第二阶段的DDPG训练提供奖励。具体而言,DDPG负责学习参与客户端的选择、客户端的CPU配置及传输功率。一种全新的感知掉队客户端的客户端关联与带宽分配(SCABA)算法可高效优化另一组决策,并为DDPG评估奖励值。实验表明,在可学习参数大幅减少的情况下,当HFL的测试精度要求为0.9时,TP-DDPG能够快速收敛到有效策略,与基准方法相比可将HFL的训练时间缩短39.4%。

英文摘要:

Federated learning (FL) is a viable technique to train a shared machine learning model without sharing data. Hierarchical FL (HFL) system has yet to be studied regrading its multiple levels of energy, computation, communication, and client scheduling, especially when it comes to clients relying on energy harvesting to power their operations. This paper presents a new two-phase deep deterministic policy gradient (DDPG) framework, referred to as ``TP-DDPG'', to balance online the learning delay and model accuracy of an FL process in an energy harvesting-powered HFL system. The key idea is that we divide optimization decisions into two groups, and employ DDPG to learn one group in the first phase, while interpreting the other group as part of the environment to provide rewards for training the DDPG in the second phase. Specifically, the DDPG learns the selection of participating clients, and their CPU configurations and the transmission powers. A new straggler-aware client association and bandwidth allocation (SCABA) algorithm efficiently optimizes the other decisions and evaluates the reward for the DDPG. Experiments demonstrate that with substantially reduced number of learnable parameters, the TP-DDPG can quickly converge to effective polices that can shorten the training time of HFL by 39.4% compared to its benchmarks, when the required test accuracy of HFL is 0.9.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑