arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2108.00668cs.ITmath.IT
收藏

多用户MISO UAV通信的三维轨迹设计:一种深度强化学习方法

Three-Dimensional Trajectory Design for Multi-User MISO UAV Communications: A Deep Reinforcement Learning Approach

Yang Wang, Zhen Gao

摘要:

本文研究了一种多用户下行多输入单输出(MISO)无人机通信系统,该系统采用多天线无人机为多个地面终端提供服务。与现有的方法只集中在一个简化的二维场景,本文认为是一个三维(3D)的城市环境中,无人机的3D轨迹的设计,以最大限度地减少数据传输完成时间的实际吞吐量和飞行运动的限制。具体来说,我们提出了一种基于深度强化学习(DRL)的完成时间最小化轨迹设计(DRL-TDCTM),它是从深度确定性策略梯度算法开发的。特别地,为了表示无人机和环境的状态信息,我们设置了附加信息,即,融合后的信息素作为奖励的参考,方便了算法设计。该算法通过与外部环境的相互作用,在相应的马尔可夫决策过程中,所提出的算法可以连续和自适应地学习如何调整无人机的运动策略。最后,仿真结果表明,所提出的DRL-TDCTM算法优于传统的基线方法。

英文摘要:

In this paper, we investigate a multi-user downlink multiple-input single-output (MISO) unmanned aerial vehicle (UAV) communication system, where a multi-antenna UAV is employed to serve multiple ground terminals. Unlike existing approaches focus only on a simplified two-dimensional scenario, this paper considers a three-dimensional (3D) urban environment, where the UAV's 3D trajectory is designed to minimize data transmission completion time subject to practical throughput and flight movement constraints. Specifically, we propose a deep reinforcement learning (DRL)-based trajectory design for completion time minimization (DRL-TDCTM), which is developed from a deep deterministic policy gradient algorithm. In particular, to represent the state information of UAV and environment, we set an additional information, i.e., the merged pheromone, as a reference of reward which facilitates the algorithm design. By interacting with the external environment in the corresponding Markov decision process, the proposed algorithm can continuously and adaptively learn how to adjust the UAV's movement strategy. Finally, simulation results show the superiority of the proposed DRL-TDCTM algorithm over the conventional baseline methods.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑