DRAG：异构网络中基于深度强化学习的基站激活
DRAG: Deep Reinforcement Learning Based Base Station Activation in Heterogeneous Networks
摘要：
异构网络（HetNet）被认为是满足前所未有的移动业务需求的关键解决方案，其中密集地部署小小区基站（SBS）以从宏基站（BS）卸载业务。高密度的SBS是为高峰交通时间设计的，并且在非高峰时间消耗不必要的大量能源。在本文中，我们提出了一种基于深度学习的SBS激活策略，激活SBS的最佳子集，以显着降低能耗，而不影响服务质量。特别是，我们将SBS开/关切换问题制定为马尔可夫决策过程，可以通过演员评论（AC）强化学习方法来解决。为了避免传统的基于表格的方法的过高的计算和存储成本，我们建议使用深度神经网络来近似AC方法中的策略和价值函数。此外，为了加快训练过程，我们采用了深度确定性策略梯度（DDPG）方法以及一种新的动作细化方案。通过大量的数值模拟，我们表明，该计划大大优于现有的方法在能源效率和计算效率。我们还表明，该计划可以扩展到大型系统的多项式复杂性的存储和计算。
英文摘要：
Heterogeneous Network (HetNet), where Small cell Base Stations (SBSs) are densely deployed to offload traffic from macro Base Stations (BSs), is identified as a key solution to meet the unprecedented mobile traffic demand. The high density of SBSs are designed for peak traffic hours and consume an unnecessarily large amount of energy during off-peak time. In this paper, we propose a deep reinforcement-learning based SBS activation strategy that activates the optimal subset of SBSs to significantly lower the energy consumption without compromising the quality of service. In particular, we formulate the SBS on/off switching problem into a Markov Decision Process that can be solved by Actor Critic (AC) reinforcement learning methods. To avoid prohibitively high computational and storage costs of conventional tabular-based approaches, we propose to use deep neural networks to approximate the policy and value functions in the AC approach. Moreover, to expedite the training process, we adopt a Deep Deterministic Policy Gradient (DDPG) approach together with a novel action refinement scheme. Through extensive numerical simulations, we show that the proposed scheme greatly outperforms the existing methods in terms of both energy efficiency and computational efficiency. We also show that the proposed scheme can scale to large system with polynomial complexities in both storage and computation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
