通信赋能的深度强化学习优化无人机辅助网络的能效
Communication-Enabled Deep Reinforcement Learning to Optimise Energy-Efficiency in UAV-Assisted Networks
AI总结:
提出通信赋能的多智能体去中心化双深度Q网络(CMAD-DDQN),通过无人机间直接协作共享遥测数据优化3D飞行轨迹,在保证覆盖性能下显著提升系统能效,较MAD-DDQN、MADDPG和随机策略分别约提升15%、65%和85%。
摘要:
无人机(UAVs)越来越多地被部署用于在网络需求增加或现有地面蜂窝基础设施出现故障的情况下,为静态和移动地面用户提供无线连接。然而,无人机受到能量限制,并面临来自共享相同频谱的附近无人机小区的干扰挑战,从而影响系统的能效(EE)。最近的方法侧重于通过优化仅服务静态地面用户的无人机轨迹来优化系统能效,而忽略了移动用户。其他一些方法则忽略了附近无人机小区的干扰影响,假设网络环境无干扰。尽管相较于集中式无人机控制,去中心化控制的研究兴趣日益增长,但无人机之间通过直接协作来改善协调同时优化系统能效的方法尚未得到充分探索。为此,我们提出了一种直接协作的通信赋能多智能体去中心化双深度Q网络(CMAD-DDQN)方法。CMAD-DDQN是一种协作算法,允许无人机通过现有的3GPP指导方针与最近邻居通信,明确共享其遥测数据。这使得智能体控制的无人机能够通过填补知识空白并收敛到最优策略来优化其3D飞行轨迹。仿真结果表明,所提方法在最大化系统能效方面优于现有基线,且未降低网络中的覆盖性能。CMAD-DDQN方法优于忽略无人机间直接协作的MAD-DDQN、考虑2D无人机部署设计而忽略附近无人机小区干扰的多智能体深度确定性策略梯度(MADDPG)以及随机策略方法,分别约提升15%、65%和85%。
英文摘要:
Unmanned aerial vehicles (UAVs) are increasingly deployed to provide wireless connectivity to static and mobile ground users in situations of increased network demand or points of failure in existing terrestrial cellular infrastructure. However, UAVs are energy-constrained and experience the challenge of interference from nearby UAV cells sharing the same frequency spectrum, thereby impacting the system's energy efficiency (EE). Recent approaches focus on optimising the system's EE by optimising the trajectory of UAVs serving only static ground users and neglecting mobile users. Several others neglect the impact of interference from nearby UAV cells, assuming an interference-free network environment. Despite growing research interest in decentralised control over centralised UAVs' control, direct collaboration among UAVs to improve coordination while optimising the systems' EE has not been adequately explored. To address this, we propose a direct collaborative communication-enabled multi-agent decentralised double deep Q-network (CMAD-DDQN) approach. The CMAD-DDQN is a collaborative algorithm that allows UAVs to explicitly share their telemetry via existing 3GPP guidelines by communicating with their nearest neighbours. This allows the agent-controlled UAVs to optimise their 3D flight trajectories by filling up knowledge gaps and converging to optimal policies. Simulation results show that the proposed approach outperforms existing baselines in terms of maximising the systems' EE without degrading coverage performance in the network. The CMAD-DDQN approach outperforms the MAD-DDQN that neglects direct collaboration among UAVs, the multi-agent deep deterministic policy gradient (MADDPG) and random policy approaches that consider a 2D UAV deployment design while neglecting interference from nearby UAV cells by about 15%, 65% and 85%, respectively.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。