arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2212.13396eess.SYcs.SY
收藏

贝叶斯优化增强的深度强化学习在多无人机网络中的轨迹规划和网络形成

Bayesian Optimization Enhanced Deep Reinforcement Learning for Trajectory Planning and Network Formation in Multi-UAV Networks

Shimin Gong, Meng Wang, Bo Gu, Wenjie Zhang, Dinh Thai Hoang, Dusit Niyato

摘要:

在本文中,我们采用多个无人机协调的基站(BS),以帮助地面用户(GU)卸载他们的传感数据。不同的无人机可以调整其轨迹和网络形成,以加快数据传输通过多跳中继。航迹规划的目标是收集所有GU的数据,而无人机的网络形成优化的多跳无人机网络拓扑,以最小化能量消耗和传输延迟。关节网络的形成和轨迹优化是通过两步迭代的方法来解决的。首先,我们设计了自适应网络形成方案,通过使用启发式算法来平衡无人机的能量消耗和数据队列大小。然后,在固定网络形成的情况下,通过使用多智能体深度强化学习,在不知道GU的交通需求和空间分布的情况下,进一步优化无人机的轨迹。为了提高学习效率,我们进一步采用贝叶斯优化估计无人机的飞行决策的基础上的历史轨迹点。这有助于避免低效的动作探索,并提高模型训练的收敛速度。仿真结果表明,无人机的航迹规划和网络形成之间存在密切的时空耦合。与几种基线相比,我们的解决方案可以更好地利用无人机在数据卸载中的合作,从而提高能量效率和延迟性能。

英文摘要:

In this paper, we employ multiple UAVs coordinated by a base station (BS) to help the ground users (GUs) to offload their sensing data. Different UAVs can adapt their trajectories and network formation to expedite data transmissions via multi-hop relaying. The trajectory planning aims to collect all GUs' data, while the UAVs' network formation optimizes the multi-hop UAV network topology to minimize the energy consumption and transmission delay. The joint network formation and trajectory optimization is solved by a two-step iterative approach. Firstly, we devise the adaptive network formation scheme by using a heuristic algorithm to balance the UAVs' energy consumption and data queue size. Then, with the fixed network formation, the UAVs' trajectories are further optimized by using multi-agent deep reinforcement learning without knowing the GUs' traffic demands and spatial distribution. To improve the learning efficiency, we further employ Bayesian optimization to estimate the UAVs' flying decisions based on historical trajectory points. This helps avoid inefficient action explorations and improves the convergence rate in the model training. The simulation results reveal close spatial-temporal couplings between the UAVs' trajectory planning and network formation. Compared with several baselines, our solution can better exploit the UAVs' cooperation in data offloading, thus improving energy efficiency and delay performance.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑