arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2408.02501cs.ITeess.SPmath.IT
收藏

通过混合控制深度强化学习实现空天地一体化网络中分层联邦边缘学习的公平资源分配

Fair Resource Allocation For Hierarchical Federated Edge Learning in Space-Air-Ground Integrated Networks via Deep Reinforcement Learning with Hybrid Control

Chong Huang, Gaojie Chen, Pei Xiao, Jonathon A. Chambers, Wei Huang

AI总结:

本文提出SAGIN中的分层联邦边缘学习框架,采用DSAC与混合动作解耦-再耦合方法联合优化资源分配和聚合权重,并设计动态奖励以保障多任务公平性。

摘要:

空天地一体化网络(SAGIN)凭借其泛在覆盖、快速灵活部署以及多层协作能力,已成为未来无线通信的关键研究方向。然而,将分层联邦学习(HFL)与边缘计算和SAGIN相融合仍是一个有待解决的复杂开放性问题。本文提出了一种在SAGIN中应用HFL的新颖框架,分别利用空中平台和低地球轨道(LEO)卫星作为边缘服务器与云服务器,从而为HFL提供多层聚合能力。所提出的系统还考虑了星间链路(ISLs)的存在,使卫星能够相互交换联邦学习模型。此外,我们考虑了需要在有限卫星服务时间内完成的多个不同计算任务。为了在确保公平性的同时最大化所有任务的收敛性能,我们提出使用分布式软演员-评论家(DSAC)算法来优化SAGIN中的资源分配以及HFL中的聚合权重。此外,我们通过解耦与再耦合方法解决了深度强化学习(DRL)中混合动作空间的效率问题,并设计了一种新的动态调整奖励函数,以确保联邦学习中多个任务之间的公平性。仿真结果证明了我们所提算法的优越性,该算法始终优于基线方法,并为解决SAGIN中高度复杂的优化问题提供了一种有前景的解决方案。

英文摘要:

The space-air-ground integrated network (SAGIN) has become a crucial research direction in future wireless communications due to its ubiquitous coverage, rapid and flexible deployment, and multi-layer cooperation capabilities. However, integrating hierarchical federated learning (HFL) with edge computing and SAGINs remains a complex open issue to be resolved. This paper proposes a novel framework for applying HFL in SAGINs, utilizing aerial platforms and low Earth orbit (LEO) satellites as edge servers and cloud servers, respectively, to provide multi-layer aggregation capabilities for HFL. The proposed system also considers the presence of inter-satellite links (ISLs), enabling satellites to exchange federated learning models with each other. Furthermore, we consider multiple different computational tasks that need to be completed within a limited satellite service time. To maximize the convergence performance of all tasks while ensuring fairness, we propose the use of the distributional soft-actor-critic (DSAC) algorithm to optimize resource allocation in the SAGIN and aggregation weights in HFL. Moreover, we address the efficiency issue of hybrid action spaces in deep reinforcement learning (DRL) through a decoupling and recoupling approach, and design a new dynamic adjusting reward function to ensure fairness among multiple tasks in federated learning. Simulation results demonstrate the superiority of our proposed algorithm, consistently outperforming baseline approaches and offering a promising solution for addressing highly complex optimization problems in SAGINs.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑