Yet another look at Harris' ergodic theorem for Markov chains
本文给出了马尔可夫链的Harris遍历定理的一个变形的初等证明。这个定理,可以追溯到50年代,基本上说,一个马尔可夫链是唯一遍历,如果它允许一个“小”集,这是访问无限频繁。这给出了一个扩展的想法Doeblin的无界状态空间设置。通常这是通过找到一个具有“小”水平集的李雅普诺夫函数来建立的。这个问题已经研究了许多作者(cf. Harris,Hasminskii,Nummelin,Meyn and Tweedie).如果李雅普诺夫函数足够强,则在加权上确界范数中存在谱间隙(参见图1)。Meyn和Tweedie)。传统的证明这一结果依赖于分解的马尔可夫链到远足远离小集和仔细分析的指数尾的长度,这些远足。还有其他的变体,它们利用了泊松方程,或致力于得到显式常数。本证明是非常直接的,而是依赖于引入一个家庭的等价加权范数索引的参数$β$,并作出适当的选择,这个参数,允许结合在一个非常基本的方式的两个成分(存在的李雅普诺夫函数和不可约),这是至关重要的,在获得一个频谱间隙。这个证明的最初动机是作者对Wasserstein度量中的谱隙的研究。本说明中的证明是我们在全变差设置中推理的一个版本,我们使用该设置来指导arXiv:math/0602479中的计算。虽然我们最初是为此目的编写这份报告的,但我们希望它本身将引起人们的兴趣。
Let $X=(X_t, t\geq 0)$ be a self-similar Markov process taking values in $\mathbb{R}$ such that the state 0 is a trap. In this paper, we present a necessary and sufficient condition for the existence of a self-similar recurrent extension of $X$ that leaves 0 continuously. The condition is expressed in terms of the associated Markov additive process via the Lamperti-Kiu representation. Our results extend those of Fitzsimmons (2006) and Rivero (2005, 2007) where the existence and uniqueness of a recurrent extension for positive self similar Markov processes were treated. In particular, we descri…
We consider a multidimensional time-homogeneous dynamical system and add a randomly perturbed time-dependent deterministic signal to some of its components, giving rise to a high-dimensional system of stochastic differential equations, which is driven by possibly very low-dimensional noise. Equations of this type commonly occur in biology when modelling neurons or in statistical mechanics for certain Hamiltonian systems. We provide verifiable conditions on the original deterministic dynamical system under which the solution to the respective stochastic system features a point in the interior o…
The paper presents new asymptotic recurrent algorithms of phase space reduction for regularly and singularly perturbed semi-Markov processes. These algorithms give effective conditions of weak convergence for distributions and convergence of expectations for hitting times as well as recurrent formulas for computing the corresponding normalisation functions, Laplace transforms for limiting distributions and limits for expectations.
The Doeblin Graph of a countable state space Markov chain describes the joint pathwise evolutions of the Markov dynamics starting from all possible initial conditions, with two paths coalescing when they reach the same point of the state space at the same time. Its Bridge Doeblin subgraph only contains the paths starting from a tagged point of the state space at all possible times. In the irreducible, aperiodic, and positive recurrent case, the following results are known: the Bridge Doeblin Graph is an infinite tree that is unimodularizable. Moreover, it contains a single bi-infinite path, wh…
In the analysis of Markov chains and processes, it is sometimes convenient to replace an unbounded state space with a "truncated" bounded state space. When such a replacement is made, one often wants to know whether the equilibrium behavior of the truncated chain or process is close to that of the untruncated system. For example, such questions arise naturally when considering numerical methods for computing stationary distributions on unbounded state space. In this paper, we use the principle of "regeneration" to show that the stationary distributions of "fixed state" truncations converge in …
We present a recurrence-transience classification for discrete-time Markov chains on manifolds with negative curvature. Our classification depends only on geometric quantities associated to the increments of the chain, defined via the Riemannian exponential map. We deduce that there exist Markov chains on a large class of such manifolds which are both recurrent and have zero average drift at every point. We give an explicit example of such a chain on hyperbolic space of arbitrary dimension, and also on a stochastically incomplete manifold. We also prove that such recurrent chains cannot be uni…
This paper discusses the irreducibility and geometric ergodicity of the Hamiltonian Monte Carlo (HMC) algorithm. We consider cases where the number of steps of the symplectic integrator is either fixed or random. Under mild conditions on the potential $\F$ associated with target distribution $π$, we first show that the Markov kernel associated to the HMC algorithm is irreducible and recurrent. Under more stringent conditions, we then establish that the Markov kernel is Harris recurrent. Finally, we provide verifiable conditions on $\F$ under which the HMC sampler is geometrically ergodic.
In this paper, the recurrent events that can occur more than one over the follow-up time have been modeled by phase-type distributions. We use the finite-state continuous-time Markov process with multi states for patients with recurrent events. The number of recurrences until time $t$, the time stay for every state and the time till death are of importances. The time till death is assumed to have a phase-type distribution (which is defined in a Markov chain environment) with interpretable parameters. The underlying continuous-time Markov chain has one absorbing state (death) and transient stat…
Using ideas borrowed from topological dynamics and ergodic theory we introduce topological and metric versions of the recurrence property for general Markov chains. The main question of interest here is how large is the set of recurrent points. We show that under some mild technical assumptions the set of non recurrent points is of zero reference measure. Necessary and sufficient conditions for a reference measure $m$ (which needs not to be dynamically invariant) to satisfy this property are obtained. These results are new even in the purely deterministic setting.
For a spectrally negative positive self-similar Markov process with an a.s. finite overall supremum we provide, in tractable detail, a kind of conditional Wiener-Hopf factorization at the maximum of the absorption time at zero, the conditioning being on the overall supremum and the jump at the overall supremum. In a companion result the Laplace transform of said absorption time (on the event that the process does not go above a given level) is identified under no other assumptions (such as the process admitting a recurrent extension and/or hitting zero continuously), generalizing some existent…
Given a non-negative Jacobi matrix describing higher order recurrence relations for multiple orthogonal polynomials of type~II and corresponding linear forms of type I, a general strategy for constructing a pair of stochastic matrices, dual to each other, is provided. The corresponding Markov chains (or 1D random walks) allow, in one transition, to reach for the N-th previous states, to remain in the state or reach for the immediately next state. The dual Markov chains allow, in one transition, to reach for the N-th next states, to remain in the state or reach for immediately previous state. T…
We develop a new algorithm for the estimation of rare event probabilities associated with the steady-state of a Markov stochastic process with continuous state space $\mathbb R^d$ and discrete time steps (i.e. a discrete-time $\mathbb R^d$-valued Markov chain). The algorithm, which we coin Recurrent Multilevel Splitting (RMS), relies on the Markov chain's underlying recurrent structure, in combination with the Multilevel Splitting method. Extensive simulation experiments are performed, including experiments with a nonlinear stochastic model that has some characteristics of complex climate mode…