arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1906.07860eess.SPcs.LGcs.NIstat.ML
收藏

IoT边缘计算中基于深度强化学习的多用户资源控制

Multi-user Resource Control with Deep Reinforcement Learning in IoT Edge Computing

Lei Lei, Huijuan Xu, Xiong Xiong, Kan Zheng, Wei Xiang, Xianbin Wang

摘要:

通过利用移动边缘计算(MEC)的概念,由大量物联网(IoT)设备生成的大量数据可以卸载到无线网络边缘的MEC服务器,以进行进一步的计算密集型处理。然而,由于物联网设备和无线网络的资源约束,需要有效地分配和调度通信和计算资源以获得更好的系统性能。本文提出了一种物联网边缘计算系统的联合计算卸载和多用户调度算法,以最小化随机流量到达下的延迟和功耗的长期平均加权和。我们制定的动态优化问题作为一个无限时域平均报酬连续时间马尔可夫决策过程(CTMDP)模型。解决多用户资源控制的MDP问题的一个关键挑战是维数灾难问题,其中MDP模型的状态空间和计算复杂度随着用户或IoT设备数量的增长而呈指数增长。为了克服这一挑战,我们使用深度强化学习(RL)技术,并提出了一种神经网络架构来近似决策后系统状态的值函数。所设计的解决CTMDP问题的算法支持基于半分布式拍卖的实现,其中IoT设备向BS提交投标以集中做出资源控制决策。仿真结果表明,该算法提供了显着的性能改善基线算法,也优于基于其他神经网络结构的RL算法。

英文摘要:

By leveraging the concept of mobile edge computing (MEC), massive amount of data generated by a large number of Internet of Things (IoT) devices could be offloaded to MEC server at the edge of wireless network for further computational intensive processing. However, due to the resource constraint of IoT devices and wireless network, both the communications and computation resources need to be allocated and scheduled efficiently for better system performance. In this paper, we propose a joint computation offloading and multi-user scheduling algorithm for IoT edge computing system to minimize the long-term average weighted sum of delay and power consumption under stochastic traffic arrival. We formulate the dynamic optimization problem as an infinite-horizon average-reward continuous-time Markov decision process (CTMDP) model. One critical challenge in solving this MDP problem for the multi-user resource control is the curse-of-dimensionality problem, where the state space of the MDP model and the computation complexity increase exponentially with the growing number of users or IoT devices. In order to overcome this challenge, we use the deep reinforcement learning (RL) techniques and propose a neural network architecture to approximate the value functions for the post-decision system states. The designed algorithm to solve the CTMDP problem supports semi-distributed auction-based implementation, where the IoT devices submit bids to the BS to make the resource control decisions centrally. Simulation results show that the proposed algorithm provides significant performance improvement over the baseline algorithms, and also outperforms the RL algorithms based on other neural network architectures.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑