arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1009.5773cs.LG
收藏

用于节能无线通信的快速强化学习

Fast Reinforcement Learning for Energy-Efficient Wireless Communications

State University of New York at Buffalo(纽约州立大学布法罗分校);University of California at Los Angeles (UCLA)(加利福尼亚大学洛杉矶分校)

Nicholas Mastronarde, Mihaela van der Schaar

摘要:

我们认为,在衰落信道上的延迟敏感数据(例如,多媒体数据)的能量有效的点对点传输的问题。关于该主题的现有研究利用以物理层为中心的解决方案,即功率控制和自适应调制和编码(AMC),或者基于动态功率管理(DMC)的系统级解决方案;然而,在延迟约束下,在存在随机和先验未知的业务和信道条件的情况下。在本报告中,我们提出了这样一个框架。我们将随机优化问题表示为马尔可夫决策过程(MDP),并使用强化学习在线求解。所提出的在线方法的优点在于:(i)它不需要业务到达和信道统计的先验知识来确定联合最优功率控制、AMC和MIMO策略;(ii)它利用了关于系统的部分信息,使得与使用传统的强化学习算法时相比需要学习更少的信息;以及(iii)它消除了对动作探索的需要,动作探索严重限制了传统强化学习算法的自适应速度和运行时性能。我们的研究结果表明,所提出的学习算法可以收敛到两个数量级的速度比一个国家的最先进的学习算法的物理层功率控制和三个数量级的速度比传统的强化学习算法。

英文摘要:

We consider the problem of energy-efficient point-to-point transmission of delay-sensitive data (e.g. multimedia data) over a fading channel. Existing research on this topic utilizes either physical-layer centric solutions, namely power-control and adaptive modulation and coding (AMC), or system-level solutions based on dynamic power management (DPM); however, there is currently no rigorous and unified framework for simultaneously utilizing both physical-layer centric and system-level techniques to achieve the minimum possible energy consumption, under delay constraints, in the presence of stochastic and a priori unknown traffic and channel conditions. In this report, we propose such a framework. We formulate the stochastic optimization problem as a Markov decision process (MDP) and solve it online using reinforcement learning. The advantages of the proposed online method are that (i) it does not require a priori knowledge of the traffic arrival and channel statistics to determine the jointly optimal power-control, AMC, and DPM policies; (ii) it exploits partial information about the system so that less information needs to be learned than when using conventional reinforcement learning algorithms; and (iii) it obviates the need for action exploration, which severely limits the adaptation speed and run-time performance of conventional reinforcement learning algorithms. Our results show that the proposed learning algorithms can converge up to two orders of magnitude faster than a state-of-the-art learning algorithm for physical layer power-control and up to three orders of magnitude faster than conventional reinforcement learning algorithms.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑