arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2202.06345cs.ITmath.IT
收藏

基于强化学习的功率控制,用于可靠的任务关键型无线传输

Reinforcement Learning Based Power Control for Reliable Mission-Critical Wireless Transmission

Chongtao Guo, Zhengchao Li, Le Liang, Geoffrey Ye Li

AI总结:

针对任务关键型快速时变信道,提出基于拉格朗日对偶的强化学习功率控制框架,并给出基于模型算法与三阶段无模型后向 Q-learning 策略,实现近最优性能。

摘要:

本文研究任务关键型应用中快速时变信道上的序贯功率分配,旨在在保证传输成功概率的同时最小化期望总功率。具体而言,本文构建了一个具有适当奖励设计的强化学习框架,使最优策略能够最大化原问题的拉格朗日函数,并证明该拉格朗日函数的最大化器具有若干良好性质。对于基于模型的情形,提出了一种快速收敛算法,用于寻找最优拉格朗日乘子,从而得到相应的最优策略。对于无模型情形,开发了一种由在线采样、离线学习和在线运行依次组成的三阶段策略,其中设计了充分利用采样信道实现的后向 Q-learning,以加速学习过程。仿真表明,所提出的强化学习框架能够从对偶角度求解原优化问题。此外,无模型策略取得了接近最优基于模型算法的性能。

英文摘要:

In this paper, we investigate sequential power allocation over fast varying channels for mission-critical applications, aiming to minimize the expected sum power while guaranteeing the transmission success probability. In particular, a reinforcement learning framework is constructed with appropriate reward design so that the optimal policy maximizes the Lagrangian of the primal problem, where the maximizer of the Lagrangian is shown to have several good properties. For the model-based case, a fast converging algorithm is proposed to find the optimal Lagrange multiplier and thus the corresponding optimal policy. For the model-free case, we develop a three-stage strategy, composed in order of online sampling, offline learning, and online operation, where a backward Q-learning with full exploitation of sampled channel realizations is designed to accelerate the learning process. According to our simulation, the proposed reinforcement learning framework can solve the primal optimization problem from the dual perspective. Moreover, the model-free strategy achieves a performance close to that of the optimal model-based algorithm.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑