arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2402.12260cs.LG
收藏

车载网络中的非正交年龄最优信息传输:一种元多目标强化学习方法

Non-orthogonal Age-Optimal Information Dissemination in Vehicular Networks: A Meta Multi-Objective Reinforcement Learning Approach

York University(约克大学);Memorial University of Newfoundland(纽芬兰纪念大学);University of the Fraser Valley(弗雷泽河谷大学)

A. A. Habob, H. Tabassum, O. Waqar

AI总结:

本文研究车载网络中最小化信息年龄与功耗的非正交信息传输问题,提出两阶段元多目标强化学习方法,利用混合DQN-DDPG模型估计高质量帕累托前沿,显著减少训练时间。

摘要:

本文考虑在车载网络中最小化信息年龄(AoI)和发射功耗,其中路边单元(RSU)向车辆提供关于一组物理过程的及时更新。我们考虑非正交多模态信息传输,其基于RSU的叠加消息传输和车辆处的串行干扰消除(SIC)。该问题被建模为多目标混合整数非线性规划问题,因此获得帕累托最优前沿极具挑战性。首先,我们利用加权和法将多目标问题分解为对应于每个预定义目标偏好权重的多个单目标子问题。然后,我们开发了一个混合深度Q网络(DQN)-深度确定性策略梯度(DDPG)模型来求解每个子问题。DQN优化解码顺序,DDPG求解连续功率分配。该模型需要为每个子问题重新训练。接着,我们提出了一种两阶段元多目标强化学习解决方案,仅需少量微调更新步骤即可估计帕累托前沿,无需为每个子问题重新训练模型。仿真结果证明了所提方案相比现有基准的有效性,且元多目标强化学习模型能以更短的训练时间估计出高质量的帕累托前沿。

英文摘要:

This paper considers minimizing the age-of-information (AoI) and transmit power consumption in a vehicular network, where a roadside unit (RSU) provides timely updates about a set of physical processes to vehicles. We consider non-orthogonal multi-modal information dissemination, which is based on superposed message transmission from RSU and successive interference cancellation (SIC) at vehicles. The formulated problem is a multi-objective mixed-integer nonlinear programming problem; thus, a Pareto-optimal front is very challenging to obtain. First, we leverage the weighted-sum approach to decompose the multi-objective problem into a set of multiple single-objective sub-problems corresponding to each predefined objective preference weight. Then, we develop a hybrid deep Q-network (DQN)-deep deterministic policy gradient (DDPG) model to solve each optimization sub-problem respective to predefined objective-preference weight. The DQN optimizes the decoding order, while the DDPG solves the continuous power allocation. The model needs to be retrained for each sub-problem. We then present a two-stage meta-multi-objective reinforcement learning solution to estimate the Pareto front with a few fine-tuning update steps without retraining the model for each sub-problem. Simulation results illustrate the efficacy of the proposed solutions compared to the existing benchmarks and that the meta-multi-objective reinforcement learning model estimates a high-quality Pareto frontier with reduced training time.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑