马尔可夫链中单个状态平稳概率的逼近
Approximating the Stationary Probability of a Single State in a Markov chain
摘要：
本文提出了一种新的迭代蒙特卡罗方法来近似正常返马尔可夫链的单个状态的平稳概率。我们利用的特征，即一个状态$i$的平稳概率是成反比的随机游走开始于$i$的预期返回时间。我们的方法得到一个$ε$ -乘法密切估计概率大于$1 - α$使用最多$\tilde{O}\left(t_{\text{mix}} \ln(1/α) / π_i ε^2 \right)$模拟随机行走步骤的马尔可夫链上的所有迭代，其中$t_{\text{mix}}$是标准的混合时间和$π_i$是平稳概率。此外，在每次迭代的估计被保证是一个高概率的上限，并在预期中随着迭代次数的减少，使我们能够监控算法的进展和设计有效的终止标准。我们提出了一个终止准则，保证$ε(1 + 4 \ln(2) t_{\text{mix}})$乘性错误性能的状态与固定概率大于$Δ$，而提供一个附加的错误与固定概率小于$Δ\in (0,1)$的状态。该算法以及该终止标准最多使用$\tilde{O}\left(\frac{\ln(1/α)}{ε^2} \min\left(\frac{t_{\text{mix}}}{π_i}, \frac{1}{εΔ}\right)\right)$模拟的随机行走步数，该步数由相对于马尔可夫链的常数限定。我们提供了一个紧密的分析，我们的算法的基础上的混合时间的局部加权的变体。通过李雅普诺夫函数分析，我们的结果自然地推广到可数无限状态空间马尔可夫链。
英文摘要：
In this paper, we present a novel iterative Monte Carlo method for approximating the stationary probability of a single state of a positive recurrent Markov chain. We utilize the characterization that the stationary probability of a state $i$ is inversely proportional to the expected return time of a random walk beginning at $i$. Our method obtains an $ε$-multiplicative close estimate with probability greater than $1 - α$ using at most $\tilde{O}\left(t_{\text{mix}} \ln(1/α) / π_i ε^2 \right)$ simulated random walk steps on the Markov chain across all iterations, where $t_{\text{mix}}$ is the standard mixing time and $π_i$ is the stationary probability. In addition, the estimate at each iteration is guaranteed to be an upper bound with high probability, and is decreasing in expectation with the iteration count, allowing us to monitor the progress of the algorithm and design effective termination criteria. We propose a termination criteria which guarantees a $ε(1 + 4 \ln(2) t_{\text{mix}})$ multiplicative error performance for states with stationary probability larger than $Δ$, while providing an additive error for states with stationary probability less than $Δ\in (0,1)$. The algorithm along with this termination criteria uses at most $\tilde{O}\left(\frac{\ln(1/α)}{ε^2} \min\left(\frac{t_{\text{mix}}}{π_i}, \frac{1}{εΔ}\right)\right)$ simulated random walk steps, which is bounded by a constant with respect to the Markov Chain. We provide a tight analysis of our algorithm based on a locally weighted variant of the mixing time. Our results naturally extend for countably infinite state space Markov chains via Lyapunov function analysis.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
