arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1801.02077cs.NI
收藏

基于异步多用户深度强化学习的无线系统切换控制

Handover Control in Wireless Systems via Asynchronous Multi-User Deep Reinforcement Learning

Zhi Wang, Lihua Li, Yue Xu, Hui Tian, Shuguang Cui

摘要:

在本文中,我们提出了一个两层的框架来学习最佳切换(HO)控制器在可能的大规模无线系统支持移动物联网(IoT)用户或传统的蜂窝用户,其中用户的移动性模式可能是异构的。特别是,我们提出的框架首先分区的用户设备(UE)与不同的移动性模式到集群,其中的移动性模式是相似的,在同一个集群。然后,在每个簇内,开发异步多用户深度强化学习方案来控制每个簇中的UE之间的HO过程,以降低HO速率同时确保一定的系统吞吐量。在该方案中,我们使用深度神经网络(DNN)作为HO控制器,每个UE通过强化学习以协作方式学习。此外,我们在执行强化学习之前使用监督学习初始化DNN控制器,以利用我们已经知道的传统HO方案,并减轻初始阶段随机探索的负面影响。此外,我们表明,所采用的基于全局参数的异步框架使我们能够更快地训练更多的UE,这可以很好地解决可扩展性问题,以支持大型系统。最后,仿真结果表明,所提出的框架可以实现更好的性能比最先进的在线计划,在HO率。

英文摘要:

In this paper, we propose a two-layer framework to learn the optimal handover (HO) controllers in possibly large-scale wireless systems supporting mobile Internet-of-Things (IoT) users or traditional cellular users, where the user mobility patterns could be heterogeneous. In particular, our proposed framework first partitions the user equipments (UEs) with different mobility patterns into clusters, where the mobility patterns are similar in the same cluster. Then, within each cluster, an asynchronous multi-user deep reinforcement learning scheme is developed to control the HO processes across the UEs in each cluster, in the goal of lowering the HO rate while ensuring certain system throughput. In this scheme, we use a deep neural network (DNN) as an HO controller learned by each UE via reinforcement learning in a collaborative fashion. Moreover, we use supervised learning in initializing the DNN controller before the execution of reinforcement learning to exploit what we already know with traditional HO schemes and to mitigate the negative effects of random exploration at the initial stage. Furthermore, we show that the adopted global-parameter-based asynchronous framework enables us to train faster with more UEs, which could nicely address the scalability issue to support large systems. Finally, simulation results demonstrate that the proposed framework can achieve better performance than the state-of-art on-line schemes, in terms of HO rates.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑