arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2407.03280cs.ITmath.IT

面向无人机辅助移动边缘计算网络的协作式多智能体深度强化学习方法

Cooperative Multi-Agent Deep Reinforcement Learning Methods for UAV-aided Mobile Edge Computing Networks

Mintae Kim, Hoon Lee, Sangwon Hwang, Merouane Debbah, Inkyu Lee

AI总结:

本文针对无人机辅助移动边缘计算网络,提出一种基于图注意力网络的协作式多智能体深度强化学习方法,通过消息动作与求解动作实现去中心化协调,并设计了可扩展训练算法以适配任意设备数量,实验证明其性能优于传统方法。

摘要:

本文提出了一种用于无人机辅助移动边缘计算(MEC)网络的协作式多智能体深度强化学习(MADRL)方法。一架具备计算能力的无人机可以向地面物联网设备(IDs)提供任务卸载服务。由于只能对整个网络状态进行部分观测,无人机和物联网设备需要分别确定各自的MEC策略,即无人机轨迹、资源分配和任务卸载策略。这要求对无人机与物联网设备之间的决策过程及协调策略进行联合优化。为解决这一难题,所提出的协作式MADRL方法计算两类动作变量,即消息动作和求解动作,每一类动作均由专用的演员神经网络(NNs)生成。因此,每个智能体能够自动封装其协调消息,以去中心化的方式提升MEC性能。所提出的演员结构基于图注意力网络设计,使得无论物联网设备数量多少均可进行操作。此外,还提出了一种可扩展的训练算法,用于针对任意网络配置训练一组神经网络。数值结果表明,所提出的协作式MADRL方法优于传统方法。

英文摘要:

This paper presents a cooperative multi-agent deep reinforcement learning (MADRL) approach for unmmaned aerial vehicle (UAV)-aided mobile edge computing (MEC) networks. An UAV with computing capability can provide task offlaoding services to ground internet-of-things devices (IDs). With partial observation of the entire network state, the UAV and the IDs individually determine their MEC strategies, i.e., UAV trajectory, resource allocation, and task offloading policy. This requires joint optimization of decision-making process and coordination strategies among the UAV and the IDs. To address this difficulty, the proposed cooperative MADRL approach computes two types of action variables, namely message action and solution action, each of which is generated by dedicated actor neural networks (NNs). As a result, each agent can automatically encapsulate its coordination messages to enhance the MEC performance in the decentralized manner. The proposed actor structure is designed based on graph attention networks such that operations are possible regardless of the number of IDs. A scalable training algorithm is also proposed to train a group of NNs for arbitrary network configurations. Numerical results demonstrate the superiority of the proposed cooperative MADRL approach over conventional methods.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑