arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2102.03077cs.ITeess.SPmath.IT
收藏

深度强化学习用于无蜂窝网络中的节能波束形成设计

Deep Reinforcement Learning for Energy-Efficient Beamforming Design in Cell-Free Networks

Weilai Li, Wanli Ni, Hui Tian, Meihui Hua

摘要:

无蜂窝网络被认为是一种满足未来无线网络更多需求的有前途的架构,其中分布式接入点与边缘云处理器协调,以共同为紧凑区域中的较少数量的用户设备提供服务。研究了无蜂窝网络中基于深度强化学习(DRL)的上行链路波束赋形设计问题,以最大化长期能量效率(EE)。首先,基于最小均方误差信道估计,利用连续干扰抵消进行信号检测,推导出信号干扰噪声比(SINR)的表达式。其次,根据信干噪比的计算公式,定义了波束形成矩阵的函数,即长期等效增益。第三,针对状态和动作空间连续的动态波束形成设计问题,利用深度确定性策略梯度(DDPG)算法的双网结构优势,提出了一种支持DRL的波束形成设计方案。最后,仿真结果表明,基于DDPG的波束形成设计能够收敛到最佳的EE性能。研究了超参数对基于DDPG的波束形成设计的EE性能的影响,结果表明,适当的折扣因子和隐层大小可以提高EE性能。

英文摘要:

Cell-free network is considered as a promising architecture for satisfying more demands of future wireless networks, where distributed access points coordinate with an edge cloud processor to jointly provide service to a smaller number of user equipments in a compact area. In this paper, the problem of uplink beamforming design is investigated for maximizing the long-term energy efficiency (EE) with the aid of deep reinforcement learning (DRL) in the cell-free network. Firstly, based on the minimum mean square error channel estimation and exploiting successive interference cancellation for signal detection, the expression of signal to interference plus noise ratio (SINR) is derived. Secondly, according to the formulation of SINR, we define the long-term EE, which is a function of beamforming matrix. Thirdly, to address the dynamic beamforming design with continuous state and action space, a DRL-enabled beamforming design is proposed based on deep deterministic policy gradient (DDPG) algorithm by taking the advantage of its double-network architecture. Finally, the results of simulation indicate that the DDPG-based beamforming design is capable of converging to the optimal EE performance. Furthermore, the influence of hyper-parameters on the EE performance of the DDPG-based beamforming design is investigated, and it is demonstrated that an appropriate discount factor and hidden layers size can facilitate the EE performance.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑