arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2005.11885eess.SPcs.LG
收藏

优化驱动的深度强化学习在IRS辅助无线通信中的稳健波束形成

Optimization-driven Deep Reinforcement Learning for Robust Beamforming in IRS-assisted Wireless Communications

Sun Yat-sen University(中山大学);Huazhong University of Science and Technology(华中科技大学);University of Technology Sydney(悉尼科技大学);Nanyang Technological University(南洋理工大学)

Jiaye Lin, Yuze Zou, Xiaoru Dong, Shimin Gong, Dinh Thai Hoang, Dusit Niyato

摘要:

智能反射面(IRS)是一种很有前途的技术,以协助下行链路信息传输从多天线接入点(AP)到接收机。在本文中,我们最小化AP的发射功率的AP的主动波束形成和IRS的被动波束形成的联合优化。由于不确定的信道条件下,我们制定了一个鲁棒的功率最小化问题的接收机的信噪比(SNR)的要求和IRS的功率预算约束。我们提出了一种深度强化学习(DRL)方法,可以根据过去的经验调整波束成形策略。为了提高学习性能,我们推导出凸逼近作为鲁棒问题的下界,并将其集成到DRL框架中,从而推广了一种新型优化驱动的深度确定性策略梯度(DDPG)方法。具体地,当DDPG算法生成动作的一部分(例如,被动波束形成),我们可以使用基于模型的凸近似来优化其它部分(例如,主动波束成形)的动作更有效。仿真结果表明,与传统的无模型DDPG算法相比,优化驱动的DDPG算法能够显著提高学习率和奖励性能.

英文摘要:

Intelligent reflecting surface (IRS) is a promising technology to assist downlink information transmissions from a multi-antenna access point (AP) to a receiver. In this paper, we minimize the AP's transmit power by a joint optimization of the AP's active beamforming and the IRS's passive beamforming. Due to uncertain channel conditions, we formulate a robust power minimization problem subject to the receiver's signal-to-noise ratio (SNR) requirement and the IRS's power budget constraint. We propose a deep reinforcement learning (DRL) approach that can adapt the beamforming strategies from past experiences. To improve the learning performance, we derive a convex approximation as a lower bound on the robust problem, which is integrated into the DRL framework and thus promoting a novel optimization-driven deep deterministic policy gradient (DDPG) approach. In particular, when the DDPG algorithm generates a part of the action (e.g., passive beamforming), we can use the model-based convex approximation to optimize the other part (e.g., active beamforming) of the action more efficiently. Our simulation results demonstrate that the optimization-driven DDPG algorithm can improve both the learning rate and reward performance significantly compared to the conventional model-free DDPG algorithm.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑