基于确定性模型的马尔可夫链不可约性和非周期性的可验证条件
Verifiable Conditions for the Irreducibility and Aperiodicity of Markov Chains by Analyzing Underlying Deterministic Models
摘要:
我们考虑马尔可夫链,服从以下一般非线性状态空间模型:$Φ_{k+1} = F(Φ_k, α(Φ_k, U_{k+1}))$其中的功能$F$是$C^1$,而$α$是典型的不连续和$\{U_k: k \in \mathbb{Z}_{> 0} \}$是一个独立和同分布的过程。我们假设对于所有的$x$,随机变量$α(x, U_1)$允许密度$p_x$,使得$(x, w) \mapsto p_x(w)$是下半连续的。我们推广和扩展以前的结果,连接属性的基础确定性控制模型提供的条件链$φ$ -不可约和非周期性。通过建立在这些结果,我们表明,如果一个秩条件的可控性矩阵满足所有$x$,有一个全局吸引状态的控制模型和$φ$ -不可约的马尔可夫链之间的存在等价。另外,在可控性矩阵秩相同的条件下,证明了稳定吸引态的存在性与链的PDMATHAMV-不可约性和非周期性之间的等价性.稳定吸引状态的概念是新的。这些结果在模型上的假设比以前的那些假设要弱得多,以前的假设要求$(x,u) \mapsto F(x,α(x,u))$是$C^\infty$(虽然它在这里可以是不连续的)。另外,在不对控制集作结构性假设的情况下,建立了$φ$的不可约性和非周期性的充分必要条件,这是新颖的--即使对于$(x,u) \mapsto F(x,α(x,u))$是$C^\infty$的马尔可夫链也是如此.我们说明,条件是很容易验证的一个非平凡的和非人工的马尔可夫链的自适应随机搜索算法的背景下产生的例子,以优化连续函数的黑箱方案。
英文摘要:
We consider Markov chains that obey the following general non-linear state space model: $Φ_{k+1} = F(Φ_k, α(Φ_k, U_{k+1}))$ where the function $F$ is $C^1$ while $α$ is typically discontinuous and $\{U_k: k \in \mathbb{Z}_{> 0} \}$ is an independent and identically distributed process. We assume that for all $x$, the random variable $α(x, U_1)$ admits a density $p_x$ such that $(x, w) \mapsto p_x(w)$ is lower semi-continuous. We generalize and extend previous results that connect properties of the underlying deterministic control model to provide conditions for the chain to be $φ$-irreducible and aperiodic. By building on those results, we show that if a rank condition on the controllability matrix is satisfied for all $x$, there is equivalence between the existence of a globally attracting state for the control model and $φ$-irreducibility of the Markov chain. Additionally, under the same rank condition on the controllability matrix, we prove that there is equivalence between the existence of a steadily attracting state and the $φ$-irreducibility and aperiodicity of the chain. The notion of steadily attracting state is new. Those results hold under considerably weaker assumptions on the model than previous ones that would require $(x,u) \mapsto F(x,α(x,u))$ to be $C^\infty$ (while it can be discontinuous here). Additionally the establishment of a necessary and sufficient condition for the $φ$-irreducibility and aperiodicity without a structural assumption on the control set is novel---even for Markov chains where $(x,u) \mapsto F(x,α(x,u))$ is $C^\infty$. We illustrate that the conditions are easy to verify on a non-trivial and non-artificial example of Markov chain arising in the context of adaptive stochastic search algorithms to optimize continuous functions in a black-box scenario.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。