能量收集虚拟小蜂窝功能分裂的动态控制:一种分布式强化学习方法
Dynamic Control of Functional Splits for Energy Harvesting Virtual Small Cells: a Distributed Reinforcement Learning Approach
摘要:
在本文中,我们提出了一种网络场景,其中仅由能量收集器和电池供电的虚拟小小区的基带过程可以在位于宏基站站点的并网边缘计算服务器中机会性地执行。本文提出了相应的能量最小化问题,并提出了多智能体强化学习(RL)方法来解决该问题,并设计了分布式模糊Q-Learning和在线Q-Learning算法。通过向独立学习者广播系统级信息来实现多个代理之间的协调。网络性能的评估证实,通过广播的协调可以实现更高的系统级增益比未协调的解决方案和累积奖励更接近离线的界限。最后,我们的分析允许评估连续状态/动作表示的学习算法在更快的收敛,更高的累积奖励和更适应不断变化的环境方面的好处。
英文摘要:
In this paper, we propose a network scenario where the baseband processes of the virtual small cells powered solely by energy harvesters and batteries can be opportunistically executed in a grid-connected edge computing server, co-located at the macro base station site. We state the corresponding energy minimization problem and propose multi-agent Reinforcement Learning (RL) to solve it. Distributed Fuzzy Q-Learning and Q-Learning on-line algorithms are tailored for our purposes. Coordination among the multiple agents is achieved by broadcasting system level information to the independent learners. The evaluation of the network performance confirms that coordination via broadcasting may achieve higher system level gains than un-coordinated solutions and cumulative rewards closer to the off-line bounds. Finally, our analysis permits to evaluate the benefits of continuous state/action representation for the learning algorithms in terms of faster convergence, higher cumulative reward and more adaptation to changing environments.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。