arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2004.02315eess.SPcs.LG
收藏

基于边缘计算的IoT网络资源分配多智能体强化学习

Multi-agent Reinforcement Learning for Resource Allocation in IoT networks with Edge Computing

Queen Mary University of London(伦敦玛丽女王大学);University of Surrey(萨里大学)

Xiaolan Liu, Jiadong Yu, Yue Gao

摘要:

为了支持虚拟现实、手机游戏和可穿戴设备等流行的物联网应用,边缘计算提供了一种低延迟的集中式云计算前端分布式计算原型。然而,由于最终用户对频谱和计算资源的大量需求以及对无线电接入技术(RAT)的频繁请求,因此卸载计算对于他们来说具有挑战性。本文将物联网边缘计算网络中的计算卸载机制与资源分配进行了研究,将其描述为一个随机博弈。这里,每个终端用户是学习代理,其观察其本地环境以学习关于本地计算或边缘计算的最优决策,其目标是通过选择其发射功率电平、RAT和子信道来最小化长期系统成本,而不知道其他最终用户的任何信息。因此,一个多智能体强化学习框架的发展,以解决随机博弈与建议的独立学习者为基础的多智能体Q-学习(IL-基于MA-Q)算法。仿真结果表明,所提出的基于IL的MA-Q算法能够有效地解决该问题,并且与其他两种基准算法相比,在集中式网关处不需要额外的信道估计开销的情况下,具有更高的能量效率。

英文摘要:

To support popular Internet of Things (IoT) applications such as virtual reality, mobile games and wearable devices, edge computing provides a front-end distributed computing archetype of centralized cloud computing with low latency. However, it's challenging for end users to offload computation due to their massive requirements on spectrum and computation resources and frequent requests on Radio Access Technology (RAT). In this paper, we investigate computation offloading mechanism with resource allocation in IoT edge computing networks by formulating it as a stochastic game. Here, each end user is a learning agent observing its local environment to learn optimal decisions on either local computing or edge computing with the goal of minimizing long term system cost by choosing its transmit power level, RAT and sub-channel without knowing any information of the other end users. Therefore, a multi-agent reinforcement learning framework is developed to solve the stochastic game with a proposed independent learners based multi-agent Q-learning (IL-based MA-Q) algorithm. Simulations demonstrate that the proposed IL-based MA-Q algorithm is feasible to solve the formulated problem and is more energy efficient without extra cost on channel estimation at the centralized gateway compared to the other two benchmark algorithms.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑