arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2011.03780cs.ITmath.IT
收藏

基于深度强化学习的时变无线下行干扰信道动态功率和波束成形设计

Deep Reinforcement Learning Based Dynamic Power and Beamforming Design for Time-Varying Wireless Downlink Interference Channel

Mengfan Liu, Rui Wang

摘要:

随着无线通信技术的高速发展,无线数据传输技术以其方便、高效的特点被广泛应用于各个领域。与常用的无线信道时不变的假设不同,本文重点研究了无线下行链路时变信道,以更接近实际情况。我们的目标是在时变信道中,在截止信干噪比(SINR)、发射功率和波束成形等约束条件下,获得最大的和速率。为了适应快速变化的信道,本文摒弃了常用的凸优化算法,采用了深度强化学习算法。从功率控制、干扰不协调和波束形成等常规措施来看,在考虑措施连续变化的同时,也不能忽视因片段流产而导致的稀疏回报问题。因此,在分析相关算法的基础上,本文提出了两种基于深度确定性的策略梯度算法(DDPG)和分层DDPG算法。对于这两种算法,为了解决离散输出,DDPG是通过将Actor-Critic算法与深度Q学习(DQN)相结合来建立的,这样它可以在不牺牲DQN带来的现有优势的情况下输出连续动作,并且还可以提高性能。此外,为了解决稀疏奖励的挑战,我们利用Meta策略的思想,分层理论的DDPG中的一个代理分为一个元控制器和一个控制器作为分层DDPG。我们的仿真结果表明,建议的DDPG和层次DDPG表现良好的覆盖,收敛和和速率性能的观点。

英文摘要:

With the high development of wireless communication techniques, it is widely used in various fields for convenient and efficient data transmission. Different from commonly used assumption of the time-invariant wireless channel, we focus on the research on the time-varying wireless downlink channel to get close to the practical situation. Our objective is to gain the maximum value of sum rate in the time-varying channel under the some constraints about cut-off signal-to-interference and noise ratio (SINR), transmitted power and beamforming. In order to adapt the rapid changing channel, we abandon the frequently used algorithm convex optimization and deep reinforcement learning algorithms are used in this paper. From the view of the ordinary measures such as power control, interference incoordination and beamforming, continuous changes of measures should be put into consideration while sparse reward problem due to the abortion of episodes as an important bottleneck should not be ignored. Therefore, with the analysis of relevant algorithms, we proposed two algorithms, Deep Deterministic Policy Gradient algorithm (DDPG) and hierarchical DDPG, in our work. As for these two algorithms, in order to solve the discrete output, DDPG is established by combining the Actor-Critic algorithm with Deep Q-learning (DQN), so that it can output the continuous actions without sacrificing the existed advantages brought by DQN and also can improve the performance. Also, to address the challenge of sparse reward, we take advantage of meta policy from the idea of hierarchical theory to divide one agent in DDPG into one meta-controller and one controller as hierarchical DDPG. Our simulation results demonstrate that the proposed DDPG and hierarchical DDPG performs well from the views of coverage, convergence and sum rate performance.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑