We consider the problem of dynamic spectrum access for network utility maximization in multichannel wireless networks. The shared bandwidth is divided into K orthogonal channels. In the beginning of each time slot, each user selects a channel and transmits a packet with a certain transmission probability. After each time slot, each user that has transmitted a packet receives a local observation indicating whether its packet was successfully delivered or not (i.e., ACK signal). The objective is a multi-user strategy for accessing the spectrum that maximizes a certain network utility in a distributed manner without online coordination or message exchanges between users. Obtaining an optimal solution for the spectrum access problem is computationally expensive in general due to the large state space and partial observability of the states. To tackle this problem, we develop a novel distributed dynamic spectrum access algorithm based on deep multi-user reinforcement leaning. Specifically, at each time slot, each user maps its current state to spectrum access actions based on a trained deep-Q network used to maximize the objective function. Game theoretic analysis of the system dynamics is developed for establishing design principles for the implementation of the algorithm. Experimental results demonstrate strong performance of the algorithm.
Iris: Deep Reinforcement Learning Driven Shared Spectrum Access Architecture for Indoor Neutral-Host Small Cells
我们考虑室内移动接入,这是当前和未来移动网络的重要用例。对于这个关键用例,我们概述了一个愿景,将基于中立主机的共享小蜂窝基础设施与用于动态共享的公共频谱池相结合,作为扩展室内小蜂窝部署和开放移动运营商生态系统的一种方式。为了实现这一愿景,我们专注于与管理共享频谱接入有关的挑战(例如,3.5 GHz US CBRS频谱)。我们提出了虹膜,一个实用的共享频谱接入架构的室内中性主机小细胞。Iris的核心是一种基于深度强化学习的动态定价机制,该机制以一种为运营商和中立主机提供激励的方式,有效地调解不同运营商对共享频谱的访问。然后,我们提出了Iris系统架构,该架构将这种动态定价机制与云RAN和RAN切片设计原则一起嵌入到为室内小型小区环境量身定制的实用中性主机设计中。使用虹膜系统的原型实现,我们提出了广泛的实验评估结果,不仅提供洞察虹膜动态定价过程及其优于替代方法,但也证明了其部署的可行性。
TinyQMIX: Distributed Access Control for mMTC via Multi-agent Reinforcement Learning
The Graduate University for Advanced Studies, SOKENDAI(综合研究大学院大学);National Institute of Informatics(国立信息学研究所);The Chinese University of Hong Kong(香港中文大学)
Shesha: Opportunistic In-network Acceleration of Asynchronous Distributed Reinforcement Learning
Leibniz University Hannover(汉诺威莱布尼茨大学);University of Duisburg-Essen(杜伊斯堡-埃森大学);University of Waterloo(滑铁卢大学);University of Massachusetts Amherst(马萨诸塞大学阿默斯特分校)
Distributed Edge Caching via Reinforcement Learning in Fog Radio Access Networks
Southeast University(东南大学);Xidian University(西安电子科技大学);Shanghai Institute of Microsystem and Information Technology, Chinese Academy of Sciences(中国科学院上海微系统与信息技术研究所);University of Oulu(奥卢大学);University of Manchester(曼彻斯特大学);Harbin Institute of Technology(哈尔滨工业大学)
Cooperative Multi-Agent Reinforcement Learning Based Distributed Dynamic Spectrum Access in Cognitive Radio Networks
National University of Defense Technology(国防科技大学);Auckland University of Technology(奥克兰理工大学);Shenzhen University(深圳大学);University of British Columbia(不列颠哥伦比亚大学)
Distributed Resource Scheduling for Large-Scale MEC Systems: A Multi-Agent Ensemble Deep Reinforcement Learning with Imitation Acceleration
Hunan Normal University(湖南师范大学);Hunan University of Commerce(湖南商学院);Northumbria University(诺森比亚大学);University of Essex(埃塞克斯大学);Queen Mary University of London(伦敦玛丽女王大学)
Heterogeneous Multi-Agent Reinforcement Learning for Distributed Channel Access in WLANs
National Mobile Communications Research Laboratory, Southeast University(东南大学国家移动通信研究中心);Purple Mountain Laboratories(紫金山实验室);College of Electronics and Information Engineering, Shenzhen University(深圳大学电子与信息工程学院);Wireless Technology Lab, 2012 Laboratories, Huawei Technologies Company Ltd.(华为技术有限公司无线技术实验室);ITP Lab, the Department of Electrical and Electronic Engineering, Imperial College London(伦敦帝国理工学院电子与电气工程系ITP实验室)