arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2212.02752cs.ITcs.NImath.IT
收藏

一种用于最小化马尔可夫源信息不确定性的索引策略

An Index Policy for Minimizing the Uncertainty-of-Information of Markov Sources

Gongpu Chen, Soung Chang Liew

AI总结:

针对多马尔可夫源经多信道更新的调度问题,本文将最小化UoI之和建模为RMAB,经松弛、解耦和梯度求解得到最优松弛策略,并取整构造适用于有界代价一般RMAB的索引策略。

摘要:

本文关注有限状态马尔可夫源的信息新鲜度,并采用信息不确定性(UoI)作为性能指标。UoI由Shannon熵度量,不仅能够刻画马尔可夫源的转移动态,还能够捕捉由最近一次观测的不同取值所导致的信息质量的不同演化。我们考虑一个信息更新系统,其中M个有限状态马尔可夫源通过m条通信信道向远程监控器传输信息。我们的目标是探索最优调度策略,以最小化马尔可夫源的UoI之和。该问题被表述为一个扰动多臂赌博机(RMAB)问题。我们对RMAB进行松弛,然后将松弛后的问题解耦为M个单赌博机问题。对单赌博机问题的分析给出了有用性质;借助这些性质,松弛问题可归约为对一个凹的分段线性函数求最大值,从而使我们能够开发一种梯度方法来求解该松弛问题并获得其最优策略。通过对松弛问题的最优策略进行向上取整,我们得到了原RMAB问题的一种索引策略。值得注意的是,所提出的索引策略具有通用性,即它适用于具有有界代价函数的一般RMAB。

英文摘要:

This paper focuses on the information freshness of finite-state Markov sources, using the uncertainty of information (UoI) as the performance metric. Measured by Shannon's entropy, UoI can capture not only the transition dynamics of the Markov source but also the different evolutions of information quality caused by the different values of the last observation. We consider an information update system with M finite-state Markov sources transmitting information to a remote monitor via m communication channels. Our goal is to explore the optimal scheduling policy to minimize the sum-UoI of the Markov sources. The problem is formulated as a restless multi-armed bandit (RMAB). We relax the RMAB and then decouple the relaxed problem into M single bandit problems. Analyzing the single bandit problem provides useful properties with which the relaxed problem reduces to maximizing a concave and piecewise linear function, allowing us to develop a gradient method to solve the relaxed problem and obtain its optimal policy. By rounding up the optimal policy for the relaxed problem, we obtain an index policy for the original RMAB problem. Notably, the proposed index policy is universal in the sense that it applies to general RMABs with bounded cost functions.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑