Whittle指数策略渐近最优性的指数收敛速度
Exponential Convergence Rate for the Asymptotic Optimality of Whittle Index Policy
摘要：
我们评估了当土匪数量增加时，不安分的马尔可夫土匪的Whittle指数策略的性能。在[30]中证明，如果bandits是可索引的并且相关的确定性系统具有全局吸引子不动点，则此性能是渐近最优的。在本文中，我们表明，在相同的条件下，收敛速度是指数的强盗的数量，除非不动点是奇异的（将在后面定义）。我们的证明是基于随机系统的确定性方程的性质：我们表明，它是一个分段仿射连续动力系统内的单纯形的经验措施的土匪。使用模拟和数值求解器，我们还调查的情况下，指数率定理的条件被违反，特别是当吸引极限环出现，或当不动点是奇异的。我们说明了我们的定理上的马尔可夫衰落信道模型，这在文献中已经得到了很好的研究。最后，我们将同步模型的结果扩展到异步模型。
英文摘要：
We evaluate the performance of Whittle index policy for restless Markovian bandits, when the number of bandits grows. It is proven in [30] that this performance is asymptotically optimal if the bandits are indexable and the associated deterministic system has a global attractor fixed point. In this paper we show that, under the same conditions, the convergence rate is exponential in the number of bandits, unless the fixed point is singular (to be defined later). Our proof is based on the nature of the deterministic equation governing the stochastic system: We show that it is a piecewise affine continuous dynamical system inside the simplex of the empirical measure of the bandits. Using simulations and numerical solvers, we also investigate the cases where the conditions for the exponential rate theorem are violated, notably when attracting limit cycles appear, or when the fixed point is singular. We illustrate our theorem on a Markovian fading channel model, which has been well studied in the literature. Finally, we extend our synchronous model results to the asynchronous model.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
