arXivDaily arXiv每日学术速递 周一至周五更新
看起来今天可以划水了
arXiv 2001.07915cs.NIcs.LGeess.SP
收藏

基于强化学习的高移动毫米波通信车-小区关联算法

Reinforcement Learning Based Vehicle-cell Association Algorithm for Highly Mobile Millimeter Wave Communication

University of Oulu(奥卢大学);Kyung Hee University(庆熙大学)

Hamza Khan, Anis Elgabli, Sumudu Samarakoon, Mehdi Bennis, Choong Seon Hong

摘要:

车联网(V2X)通信是一个不断增长的通信领域,具有各种用例。研究了毫米波通信网络中的车辆-小区关联问题。目标是最大化每个车辆用户(VUE)的时间平均速率,同时确保所有VUE的目标最低速率以及低信令开销。我们首先制定的用户(车辆)关联问题作为一个离散的非凸优化问题。然后,通过利用机器学习的工具,特别是分布式深度强化学习(DDRL)和异步演员评论算法(A3C),我们提出了一个低复杂度的算法,近似于所提出的优化问题的解决方案。所提出的基于DDRL的算法赋予每个路边单元(RSU)与本地RL代理,选择一个本地的行动的基础上观察到的输入状态。不同RSU的动作被转发到中央实体,中央实体计算全局奖励,然后反馈给RSU。结果表明,每个独立训练的RL执行车辆RSU关联动作,与运行在线复杂算法来解决非凸优化问题相比,具有较低的控制开销和较低的计算复杂性。最后,仿真结果表明,与几种基准设计相比,该方案在总速率方面获得了15%的增益,VUE中断减少了20%.

英文摘要:

Vehicle-to-everything (V2X) communication is a growing area of communication with a variety of use cases. This paper investigates the problem of vehicle-cell association in millimeter wave (mmWave) communication networks. The aim is to maximize the time average rate per vehicular user (VUE) while ensuring a target minimum rate for all VUEs with low signaling overhead. We first formulate the user (vehicle) association problem as a discrete non-convex optimization problem. Then, by leveraging tools from machine learning, specifically distributed deep reinforcement learning (DDRL) and the asynchronous actor critic algorithm (A3C), we propose a low complexity algorithm that approximates the solution of the proposed optimization problem. The proposed DDRL-based algorithm endows every road side unit (RSU) with a local RL agent that selects a local action based on the observed input state. Actions of different RSUs are forwarded to a central entity, that computes a global reward which is then fed back to RSUs. It is shown that each independently trained RL performs the vehicle-RSU association action with low control overhead and less computational complexity compared to running an online complex algorithm to solve the non-convex optimization problem. Finally, simulation results show that the proposed solution achieves up to 15\% gains in terms of sum rate and 20\% reduction in VUE outages compared to several baseline designs.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑