arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1704.02613cs.NI
收藏

分布式动态频谱接入的深度多用户强化学习

Deep Multi-User Reinforcement Learning for Distributed Dynamic Spectrum Access

Oshri Naparstek, Kobi Cohen

摘要:

研究了多信道无线网络中网络效用最大化的动态频谱接入问题。共享带宽被划分为K个正交信道。在每个时隙的开始,每个用户选择一个信道并以一定的传输概率发送一个分组。在每个时隙之后,已经发送分组的每个用户接收指示其分组是否被成功递送的本地观测(即,ACK信号)。我们的目标是一个多用户的频谱访问策略,最大限度地提高了一定的网络效用,在分布式的方式,没有在线协调或用户之间的消息交换。由于频谱接入问题的状态空间较大,且状态具有部分可观测性,因此获得频谱接入问题的最优解通常在计算上是昂贵的。针对这一问题,提出了一种基于深度多用户强化学习的分布式动态频谱接入算法。具体地,在每个时隙,每个用户基于用于最大化目标函数的训练的深度Q网络将其当前状态映射到频谱接入动作。博弈论分析的系统动力学建立设计原则的算法的实施。实验结果表明该算法具有较强的性能。

英文摘要:

We consider the problem of dynamic spectrum access for network utility maximization in multichannel wireless networks. The shared bandwidth is divided into K orthogonal channels. In the beginning of each time slot, each user selects a channel and transmits a packet with a certain transmission probability. After each time slot, each user that has transmitted a packet receives a local observation indicating whether its packet was successfully delivered or not (i.e., ACK signal). The objective is a multi-user strategy for accessing the spectrum that maximizes a certain network utility in a distributed manner without online coordination or message exchanges between users. Obtaining an optimal solution for the spectrum access problem is computationally expensive in general due to the large state space and partial observability of the states. To tackle this problem, we develop a novel distributed dynamic spectrum access algorithm based on deep multi-user reinforcement leaning. Specifically, at each time slot, each user maps its current state to spectrum access actions based on a trained deep-Q network used to maximize the objective function. Game theoretic analysis of the system dynamics is developed for establishing design principles for the implementation of the algorithm. Experimental results demonstrate strong performance of the algorithm.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑