基于多Agent深度强化学习的车载云网络计算资源分配
Multi-Agent Deep Reinforcement Learning enabled Computation Resource Allocation in a Vehicular Cloud Network
摘要：
在本文中，我们研究了计算资源分配问题，在一个分布式的Ad-Hoc车辆网络没有集中的基础设施的支持。为了支持这种车辆网络中不断增长的计算需求，形成了分布式虚拟云网络（VCN），在此基础上，提出了通过在附近车辆之间卸载的计算资源共享方案。针对虚拟通信网络中计算资源时变的特点，详细分析了计算资源的统计分布特性。在此基础上，提出了一种资源感知的组合优化目标机制。为了缓解VCN中典型的多Agent环境所造成的非平稳环境，我们采用了集中训练和分散执行的框架。此外，针对目标优化问题，我们将其建模为马尔可夫博弈，并提出了基于DRL的多智能体深度确定性强化学习（MADDPG）算法来解决。有趣的是，为了克服VCN中缺乏真正的中央控制单元的困境，分配实际上是以分布式方式在车辆上完成的。仿真结果证明了该方案的有效性。
英文摘要：
In this paper, we investigate the computational resource allocation problem in a distributed Ad-Hoc vehicular network with no centralized infrastructure support. To support the ever increasing computational needs in such a vehicular network, the distributed virtual cloud network (VCN) is formed, based on which a computational resource sharing scheme through offloading among nearby vehicles is proposed. In view of the time-varying computational resource in VCN, the statistical distribution characteristics for computational resource are analyzed in detail. Thereby, a resource-aware combinatorial optimization objective mechanism is proposed. To alleviate the non-stationary environment caused by the typically multi-agent environment in VCN, we adopt a centralized training and decentralized execution framework. In addition, for the objective optimization problem, we model it as a Markov game and propose a DRL based multi-agent deep deterministic reinforcement learning (MADDPG) algorithm to solve it. Interestingly, to overcome the dilemma of lacking a real central control unit in VCN, the allocation is actually completed on the vehicles in a distributed manner. The simulation results are presented to demonstrate our scheme's effectiveness.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
