最小化蜂窝V2X通信中信息年龄的深度确定性策略梯度
Deep Deterministic Policy Gradient to Minimize the Age of Information in Cellular V2X Communications
摘要：
本文研究了蜂窝车对万物通信中的信息年龄最小化问题。为了提供最小的AoI和高可靠性的车辆的安全信息，NOMA被利用。我们重新制定了一个资源分配问题，涉及半双工收发器的选择，广播覆盖优化，功率分配和资源块调度。首先，为了得到最优解，我们将问题转化为一个混合整数非线性规划问题，并研究了其NP-困难性。NP难的结果促使我们设计简单的解决方案。因此，我们将问题建模为单智能体马尔可夫决策过程，以使用指纹深度强化学习技术（如深度Q网络（DQN）方法）有效地解决问题。然而，由于包含离散和连续优化决策的大型混合动作空间所隐含的维数灾难，应用DQN并不简单。因此，为了有效地解决这个混合离散/连续问题，简单而优雅，我们提出了一种分解技术，包括首先使用基于最先进的稳定室友匹配的匹配算法解决离散子问题，然后使用基于深度确定性策略梯度DDPG的DRL算法解决连续子问题。我们验证我们提出的方法，通过蒙特卡罗模拟，我们表明，分解匹配和DRL算法成功地最大限度地减少了AoI，并实现了近66%的性能增益相比，各种车辆的速度，传输功率，或数据包大小的最佳基准。此外，我们证明了广播覆盖的最佳值的存在，在该最佳值的学习算法提供了最佳的AoI。
英文摘要：
This paper studies the problem of minimizing the age of information (AoI) in cellular vehicle-to-everything communications. To provide minimal AoI and high reliability for vehicles' safety information, NOMA is exploited. We reformulate a resource allocation problem that involves half-duplex transceiver selection, broadcast coverage optimization, power allocation, and resource block scheduling. First, to obtain the optimal solution, we formulate the problem as a mixed-integer nonlinear programming problem and then study its NP-hardness. The NP-hardness result motivates us to design simple solutions. Consequently, we model the problem as a single-agent Markov decision process to solve the problem efficiently using fingerprint deep reinforcement learning techniques such as deep-Q-network (DQN) methods. Nevertheless, applying DQN is not straightforward due to the curse of dimensionality implied by the large and mixed action space that contains discrete and continuous optimization decisions. Therefore, to solve this mixed discrete/continuous problem efficiently, simply and elegantly, we propose a decomposition technique that consists of first solving the discrete subproblem using a matching algorithm based on state-of-the-art stable roommate matching and then solving the continuous subproblem using DRL algorithm that is based on deep deterministic policy gradient DDPG. We validate our proposed method through Monte Carlo simulations where we show that the decomposed matching and DRL algorithm successfully minimizes the AoI and achieves almost 66% performance gain compared to the best benchmarks for various vehicles' speeds, transmission power, or packet sizes. Further, we prove the existence of an optimal value of broadcast coverage at which the learning algorithm provides the optimal AoI.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
