具有Doeblin条件的隐藏随机博弈中的均匀值逼近
Approximating the Uniform Value in Hidden Stochastic Games with Doeblin Condition
AI总结:
研究隐藏随机博弈中均匀值的存在性与逼近问题,通过推广Doeblin条件证明均匀值存在并给出逼近算法,但无法精确计算。
摘要:
我们研究\emph{零和两人隐藏随机博弈},其中玩家接收部分状态观测。我们关注分析长期随机博弈的核心解概念:\emph{均匀值},即双方玩家在足够长的时间内都能保证的极限平均收益。一般情况下,先前的工作提供了没有均匀值的博弈例子。此外,对于确实具有均匀值的博弈子类,不存在逼近它的算法。因此,我们将马尔可夫链的\emph{Doeblin条件}(保证存在唯一不变测度)推广到隐藏随机博弈。非正式地说,隐藏随机博弈的Doeblin条件要求:对于每种博弈方式,存在一个固定信念,使得无论初始信念关于博弈状态如何,经过足够多的阶段后,后验信念很可能接近这个固定信念。在Doeblin条件下,我们证明了均匀值的存在性,提供了逼近它的算法,并证明没有算法可以精确计算它。然后,我们识别了转移函数上的结构条件,确保Doeblin条件在盲设置(观测无信息)和隐藏设置(观测部分信息)中都成立。当考虑只有一个玩家的博弈时,即部分可观测马尔可夫决策过程,我们的结果提供了一个新的子类,其中均匀值存在且可逼近,但不能精确计算。
英文摘要:
We study \emph{zero-sum two-player hidden stochastic games}, where players receive partial observations of the state. We focus on a central solution concept for analyzing long-duration stochastic games: the \emph{uniform value}, a limiting average payoff that both players can guarantee for sufficiently long durations. In the general case, prior work provides examples of games that do not have a uniform value. Moreover, for the subclass of games that do have a uniform value, there exists no algorithm that approximates it. Therefore, we generalize the \emph{Doeblin condition} for Markov chains (which guarantees the existence of a unique invariant measure) to hidden stochastic games. Informally, the Doeblin condition for hidden stochastic games requires that, for every way to play the game, there exists a fixed belief such that, no matter the initial belief over the state of the game, after sufficiently many stages, the posterior belief is probably close to this fixed belief. Under the Doeblin condition, we prove the existence of the uniform value, provide an algorithm to approximate it, and prove that no algorithm can compute it exactly. Then, we identify structural conditions on the transition function that ensure the Doeblin condition holds both in the blind setting, where observations are uninformative, and in the hidden setting, where observations are partially informative. When considering games with only one player, namely partially observable Markov decision processes, our results provide a novel subclass in which the uniform value exists and can be approximated, but cannot be computed exactly
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。