This paper generalizes the work of Kendall [Electron. Comm. Probab. 9 (2004) 140--151], which showed that perfect simulation, in the form of dominated coupling from the past, is always possible (although not necessarily practical) for geometrically ergodic Markov chains. Here, we consider the more general situation of positive recurrent chains and explore when it is possible to produce such a simulation algorithm for these chains. We introduce a class of chains which we name tame, for which we show that perfect simulation is possible.
In this paper, the weak convergence of impulsive recurrent process with semi-Markov switching in the scheme of Levy approximation is proved. Singular perturbation problem for the compensating operator of the extended Markov renewal process is used to prove the relative compactness.
In this paper, the weak convergence of impulsive recurrent process with Markov switching in the scheme of Levy approximation is proved. For the relative compactness, a method proposed by R. Liptser for semimartingales is used with a modification, where we apply a solution of a singular perturbation problem instead of an ergodic theorem.
Let $X=(X_t, t\geq 0)$ be a self-similar Markov process taking values in $\mathbb{R}$ such that the state 0 is a trap. In this paper, we present a necessary and sufficient condition for the existence of a self-similar recurrent extension of $X$ that leaves 0 continuously. The condition is expressed in terms of the associated Markov additive process via the Lamperti-Kiu representation. Our results extend those of Fitzsimmons (2006) and Rivero (2005, 2007) where the existence and uniqueness of a recurrent extension for positive self similar Markov processes were treated. In particular, we descri…
We prove that a positive self-similar Markov process $(X,\mathbb{P})$ that hits 0 in a finite time admits a self-similar recurrent extension that leaves 0 continuously if and only if the underlying Lévy process satisfies Cramér's condition.
In our monograph with B. Roynette and M. Yor, we construct a sigma-finite measure related to penalisations of different stochastic processes, including the Brownian motion in dimension 1 or 2, and a large class of linear diffusions. In the last chapter of the monograph, we define similar measures from recurrent Markov chains satisfying some technical conditions. In the present paper, we give a classification of these measures, in function of the minimal Martin boundary of the Markov chain considered at the beginning. We apply this classification to the examples considered at the end of our mon…
In this paper, we investigate the properties of recurrent planar Markov random walks. More precisely, we study the set of recurrent points with the use of local limit theorems. The Nagaev-Guivarc'h spectral method provides several examples for which these local limit theorems are satisfied as soon as the (standard or non-standard) central limit theorem holds.
The paper presents new asymptotic recurrent algorithms of phase space reduction for regularly and singularly perturbed semi-Markov processes. These algorithms give effective conditions of weak convergence for distributions and convergence of expectations for hitting times as well as recurrent formulas for computing the corresponding normalisation functions, Laplace transforms for limiting distributions and limits for expectations.
The Doeblin Graph of a countable state space Markov chain describes the joint pathwise evolutions of the Markov dynamics starting from all possible initial conditions, with two paths coalescing when they reach the same point of the state space at the same time. Its Bridge Doeblin subgraph only contains the paths starting from a tagged point of the state space at all possible times. In the irreducible, aperiodic, and positive recurrent case, the following results are known: the Bridge Doeblin Graph is an infinite tree that is unimodularizable. Moreover, it contains a single bi-infinite path, wh…
Importance sampling is a variance reduction technique for efficient estimation of rare-event probabilities by Monte Carlo. In standard importance sampling schemes, the system is simulated using an a priori fixed change of measure suggested by a large deviation lower bound analysis. Recent work, however, has suggested that such schemes do not work well in many situations. In this paper we consider dynamic importance sampling in the setting of uniformly recurrent Markov chains. By ``dynamic'' we mean that in the course of a single simulation, the change of measure can depend on the outcome of th…
We study positive recurrence and transience of a two-station network in which the behavior of the server in each station is governed by a Markov chain with a finite number of server states; this service process can represent various service disciplines such as a non-preemptive priority service and K-limited service. Assuming that exogenous customers arrive according to independent Markovian arrival processes (MAPs), we represent the behavior of the whole network as a continuous-time Markov chain and, by the uniformization technique, obtain the corresponding discrete-time Markov chain, which is…
We present a recurrence-transience classification for discrete-time Markov chains on manifolds with negative curvature. Our classification depends only on geometric quantities associated to the increments of the chain, defined via the Riemannian exponential map. We deduce that there exist Markov chains on a large class of such manifolds which are both recurrent and have zero average drift at every point. We give an explicit example of such a chain on hyperbolic space of arbitrary dimension, and also on a stochastically incomplete manifold. We also prove that such recurrent chains cannot be uni…
Let $\{X_n\}$ be a Markov chain with transition probability $p_{ij}=a_{j-(i-1)^+},\forall i,j\ge 0$, where $a_j=0$ provided $j<0$, $a_0>0$, $a_0+a_1<1$ and $\sum_{n=0}^\infty a_n=1$. Let $μ=\sum_{n=1}^\infty na_n$. It's known that $\{X_n\}$ is positive recurrent when $μ<1$; is null recurrent when $μ=1$; and is transient when $μ>1$. In this paper, we shall discuss the first returning speed and the last exit speed more precisely by means of $\{a_n\}$
Let $(M_{n},S_{n})_{n\ge 0}$ be a Markov random walk with positive recurrent driving chain $(M_{n})_{n\ge 0}$ having countable state space $\mathcal{S}$ and stationary distribution $π$. It is shown in this note that, if the dual sequence $({}^{\#}M_{n},{}^{\#}S_{n})_{n\ge 0}$ is positive divergent, i.e. ${}^{\#}S_{n}\to\infty$ a.s., then the strictly ascending ladder epochs $σ_{n}^{>}$ of $(M_{n},S_{n})_{n\ge 0}$ are a.s. finite and the ladder chain $(M_{σ_{n}^{>}})_{n\ge 0}$ is positive recurrent on some $\mathcal{S}^{>}\subset\mathcal{S}$. We also provide simple expressions for its stationar…
We investigate recurrence and transience of Branching Markov Chains (BMC) in discrete time. Branching Markov Chains are clouds of particles which move (according to an irreducible underlying Markov Chain) and produce offspring independently. The offspring distribution can depend on the location of the particle. If the offspring distribution is constant for all locations, these are Tree-Indexed Markov chains in the sense of \cite{benjamini94}. Starting with one particle at location $x$, we denote by $α(x)$ the probability that $x$ is visited infinitely often by the cloud. Due to the irreducibil…
Let X={X_n:n=0,1,2,...} be an irreducible, positive recurrent Markov chain with invariant probability measure π. We show that if X satisfies a one-step minorization condition, then πcan be represented as an infinite mixture. The distributions in the mixture are associated with the hitting times on an accessible atom introduced via the splitting construction of Athreya and Ney [Trans. Amer. Math. Soc. 245 (1978) 493-501] and Nummelin [Z. Wahrsch. Verw. Gebiete 43 (1978) 309-318]. When the small set in the minorization condition is the entire state space, our mixture representation of πreduces t…
It is well known that for a strictly stationary, reversible, Harris recurrent Markov chain, the $ρ$-mixing condition is equivalent to geometric ergodicity and to a "spectral gap" condition. In this note, it will be shown with an example that for that class of Markov chains, the "interlaced" variant of the $ρ$-mixing condition fails to be equivalent to those conditions.
We classify the possible behaviors of a class of one-dimensional stochastic recurrent growth models. In our main result, we obtain nearly optimal bounds for the tail of hitting times of some compact sets. If the process is an aperiodic irreducible Markov chain, we determine whether it is null recurrent or positive recurrent and in the latter case, we obtain a subgeometric convergence of its transition kernel to its invariant measure. We apply our results in particular to state-dependent Galton-Watson processes and we give precise estimates of the tail of the extinction time.
A $ϕ$-irreducible and aperiodic Markov chain with stationary probability distribution will converge to its stationary distribution from almost all starting points. The property of Harris recurrence allows us to replace ``almost all'' by ``all,'' which is potentially important when running Markov chain Monte Carlo algorithms. Full-dimensional Metropolis--Hastings algorithms are known to be Harris recurrent. In this paper, we consider conditions under which Metropolis-within-Gibbs and trans-dimensional Markov chains are or are not Harris recurrent. We present a simple but natural two-dimensional…
For many probability distributions of interest, it is quite difficult to obtain samples efficiently. Often, Markov chains are employed to obtain approximately random samples from these distributions. The primary drawback to traditional Markov chain methods is that the mixing time of the chain is usually unknown, which makes it impossible to determine how close the output samples are to having the target distribution. Here we present a new protocol, the randomness recycler (RR), that overcomes this difficulty. Unlike classical Markov chain approaches, an RR-based algorithm creates samples drawn…
If $(C_n)$ is a Markov chain on a discrete state space ${\mathcal{S}}$, a Markov chain $(C_n,M_n)$ on the product space ${\mathcal{S}}\times{\mathcal{S}}$, the cat and mouse Markov chain, is constructed. The first coordinate of this Markov chain behaves like the original Markov chain and the second component changes only when both coordinates are equal. The asymptotic properties of this Markov chain are investigated. A representation of its invariant measure is, in particular, obtained. When the state space is infinite it is shown that this Markov chain is in fact null recurrent if the initial…
In this paper, we give quantitative bounds on the $f$-total variation distance from convergence of an Harris recurrent Markov chain on an arbitrary under drift and minorisation conditions implying ergodicity at a sub-geometric rate. These bounds are then specialized to the stochastically monotone case, covering the case where there is no minimal reachable element. The results are illustrated on two examples from queueing theory and Markov Chain Monte Carlo.
We are interested in the asymptotic behavior of Markov chains on the set of positive integers for which, loosely speaking, large jumps are rare and occur at a rate that behaves like a negative power of the current state, and such that small positive and negative steps of the chain roughly compensate each other. If $X_{n}$ is such a Markov chain started at $n$, we establish a limit theorem for $\frac{1}{n}X_{n}$ appropriately scaled in time, where the scaling limit is given by a nonnegative self-similar Markov process. We also study the asymptotic behavior of the time needed by $X_{n}$ to reach…
In this paper, the recurrent events that can occur more than one over the follow-up time have been modeled by phase-type distributions. We use the finite-state continuous-time Markov process with multi states for patients with recurrent events. The number of recurrences until time $t$, the time stay for every state and the time till death are of importances. The time till death is assumed to have a phase-type distribution (which is defined in a Markov chain environment) with interpretable parameters. The underlying continuous-time Markov chain has one absorbing state (death) and transient stat…
Suppose $ E$ is a space with a null-recurrent Markov kernel $ P$. Furthermore, suppose there are infinite particles with variable weights on $ E$ performing a random walk following $ P$. Let $ X_{t}$ be a weighted functional of the position of particles at time $ t$. Under some conditions on the initial distribution of the particles the process $ (X_{t})$ is stationary over time. Non-Gaussian infinitely divisible (ID) distributions turn out to be natural candidates for the initial distribution and then the process $ (X_{t})$ is ID. We prove a functional large and moderate deviation principle f…
We give recurrence and transience criteria for two cases of time-homogeneous Markov chains on the real line with transition kernel $p(x,dy)=f_x(y-x)dy$, where $f_x(y)$ are probability densities of symmetric distributions and, for large $|y|$, have a power-law decay with exponent $α(x)+1$, with $α(x)\in(0,2)$. If $f_x(y)$ is the density of a symmetric $α$-stable distribution for negative $x$ and the density of a symmetric $β$-stable distribution for non-negative $x$, where $α,β\in(0,2)$, then the chain is recurrent if and only if $α+β\geq2.$ If the function $x\longmapsto f_x$ is periodic and if…
We consider a Markov chain on $R^+$ with asymptotically zero drift and finite second moments of jumps which is positive recurrent. A power-like asymptotic behaviour of the invariant tail distribution is proven; such a heavy-tailed invariant measure happens even if the jumps of the chain are bounded. Our analysis is based on test functions technique and on construction of a harmonic function.
Using ideas borrowed from topological dynamics and ergodic theory we introduce topological and metric versions of the recurrence property for general Markov chains. The main question of interest here is how large is the set of recurrent points. We show that under some mild technical assumptions the set of non recurrent points is of zero reference measure. Necessary and sufficient conditions for a reference measure $m$ (which needs not to be dynamically invariant) to satisfy this property are obtained. These results are new even in the purely deterministic setting.
In this paper, we are devoted to developing matrix-analytic methods for solving Poisson's equation for irreducible and positive recurrent discrete-time Markov chains (DTMCs). Two special solutions, including the deviation matrix D and the expected additive-type functional matrix K, will be considered. The results are applied to Markov chains of GI/G/1-type and MAP/G/1 queues with negative customers. Further extensions to continuous-time Markov chains (CTMCs) are also investigated.
This paper proposes a new type of recurrence where we divide the Markov chains into intervals that start when the chain enters into a subset A, then sample another subset B far away from A and end when the chain again return to A. The length of these intervals have the same distribution and if A and B are far apart, almost independent of each other. A and B may be any subsets of the state space that are far apart of each other and such that the movement between the subsets is repeated several times in a long Markov chain. The expected length of the intervals is used in a function that describe…
It is known that random walk Metropolis algorithms with heavy-tailed target densities can model atypical (slow) growth of maxima, which in general is exhibited by processes with the extremal index zero. The asymptotics of maxima of such sequences can be analyzed in terms of continuous phantom distribution functions. We show that in a large class of positive Harris recurrent Markov chains (containing the above Metropolis chains) a phantom distribution function can be recovered by starting "at the point" rather than from the stationary distribution.
Bounding chains are a technique that offers three benefits to Markov chain practitioners: a theoretical bound on the mixing time of the chain under restricted conditions, experimental bounds on the mixing time of the chain that are provably accurate and construction of perfect sampling algorithms when used in conjunction with protocols such as coupling from the past. Perfect sampling algorithms generate variates exactly from the target distribution without the need to know the mixing time of a Markov chain at all. We present here the basic theory and use of bounding chains for several chains f…
In this paper we consider two-point unilateral Markov fields on a two-dimensional lattice as considered by Pickard, Galbraith and Walley. We show that, under various ergodicity conditions, they can be perfectly simulated in the stationary state on any finite window. The techniques which are used connect perfect simulation with oriented percolation through suitable coupling constructions.
This paper addresses the question when the underlying Markov process of a multiclass queueing network is positive Harris recurrent. It is well-known that stability of the fluid limit model is a sufficient condition for this. Hence, stability of fluid (limit) models is of vital interest. Recently, it has been shown that if the fluid model satisfies certain properties, it is stable if and only if there exists a Lyapunov function. In this paper a new method is provided to conclude that the underlying Markov process is positive Harris recurrent if the fluid model is stable by using explicitly the …
Perfect sampling is a technique that uses coupling arguments to provide a sample from the stationary distribution of a Markov chain in a finite time without ever computing the distribution. This technique is very efficient if all the events in the system have monotonicity property. However, in the general (non-monotone) case, this technique needs to consider the whole state space, which limits its application only to chains with a state space of small cardinality. We propose here a new approach for the general case that only needs to consider two trajectories. Instead of the original chain, we…
We derive a sufficient condition for a $k$-th order homogeneous Markov chain $\mathbf{Z}$ with finite alphabet $\mathcal{Z}$ to have a unique invariant distribution on $\mathcal{Z}^k$. Specifically, let $\mathbf{X}$ be a first-order, stationary Markov chain with finite alphabet $\mathcal{X}$ and a single recurrent class, let $g{:}\ \mathcal{X}\to\mathcal{Z}$ be non-injective, and define the (possibly non-Markovian) process $\mathbf{Y}:=g(\mathbf{X})$ (where $g$ is applied coordinate-wise). If $\mathbf{Z}$ is the $k$-th order Markov approximation of $\mathbf{Y}$, its invariant distribution is u…
The classic arcsine law for the number $N_{n}^{>}:=n^{-1}\sum_{k=1}^{n}\mathbf{1}_{\{S_{k}>0\}}$ of positive terms, as $n\to\infty$, in an ordinary random walk $(S_{n})_{n\ge 0}$ is extended to the case when this random walk is governed by a positive recurrent Markov chain $(M_{n})_{n\ge 0}$ on a countable state space $\mathcal{S}$, that is, for a Markov random walk $(M_{n},S_{n})_{n\ge 0}$ with positive recurrent discrete driving chain. More precisely, it is shown that $n^{-1}N_{n}^{>}$ converges in distribution to a generalized arcsine law with parameter $ρ\in [0,1]$ (the classic arcsine law…
The theory of $L^2$-spectral gaps for reversible Markov chains has been studied by many authors. In this paper we consider positive recurrent general state space Markov chains with stationary transition probabilities. Replacing the assumption of reversibility by a less strong one, we still obtain a simple necessary and sufficient condition for the spectral gap property of the associated Markov operator in terms of isoperimetric constant. Moreover, we define a new sequence of isoperimetric constants which provides a necessary and sufficient condition for the existence of a spectral gap in a ver…
The question of recurrence and transience of branching Markov chains is more subtle than for ordinary Markov chains; they can be classified in transience, weak recurrence, and strong recurrence. We review criteria for transience and weak recurrence and give several new conditions for weak recurrence and strong recurrence. These conditions make a unified treatment of known and new examples possible and provide enough information to distinguish between weak and strong recurrence. This represents a step towards a general classification of branching Markov chains. In particular, we show that in \e…
For a spectrally negative positive self-similar Markov process with an a.s. finite overall supremum we provide, in tractable detail, a kind of conditional Wiener-Hopf factorization at the maximum of the absorption time at zero, the conditioning being on the overall supremum and the jump at the overall supremum. In a companion result the Laplace transform of said absorption time (on the event that the process does not go above a given level) is identified under no other assumptions (such as the process admitting a recurrent extension and/or hitting zero continuously), generalizing some existent…
Given a non-negative Jacobi matrix describing higher order recurrence relations for multiple orthogonal polynomials of type~II and corresponding linear forms of type I, a general strategy for constructing a pair of stochastic matrices, dual to each other, is provided. The corresponding Markov chains (or 1D random walks) allow, in one transition, to reach for the N-th previous states, to remain in the state or reach for the immediately next state. The dual Markov chains allow, in one transition, to reach for the N-th next states, to remain in the state or reach for immediately previous state. T…
We describe a new algorithm for the perfect simulation of variable length Markov chains and random systems with perfect connections. This algorithm, which generalizes Propp and Wilson's simulation scheme, is based on the idea of coupling into and from the past. It improves on existing algorithms by relaxing the conditions on the kernel and by accelerating convergence, even in the simple case of finite order Markov chains. Although chains of variable or infinite order have been widely investigated for decades, their use in applied probability, from information theory to bio-informatics and ling…
We consider the extreme value statistics of centrally-biased random walks with asymptotically-zero drift in the ergodic regime. We fully characterize the asymptotic distribution of the maximum for this class of Markov chains lacking translational invariance, with a particular emphasis on the relation between the time scaling of the expected value of the maximum and the stationary distribution of the process.
This paper discusses the irreducibility and geometric ergodicity of the Hamiltonian Monte Carlo (HMC) algorithm. We consider cases where the number of steps of the symplectic integrator is either fixed or random. Under mild conditions on the potential $\F$ associated with target distribution $π$, we first show that the Markov kernel associated to the HMC algorithm is irreducible and recurrent. Under more stringent conditions, we then establish that the Markov kernel is Harris recurrent. Finally, we provide verifiable conditions on $\F$ under which the HMC sampler is geometrically ergodic.
We develop a new algorithm for the estimation of rare event probabilities associated with the steady-state of a Markov stochastic process with continuous state space $\mathbb R^d$ and discrete time steps (i.e. a discrete-time $\mathbb R^d$-valued Markov chain). The algorithm, which we coin Recurrent Multilevel Splitting (RMS), relies on the Markov chain's underlying recurrent structure, in combination with the Multilevel Splitting method. Extensive simulation experiments are performed, including experiments with a nonlinear stochastic model that has some characteristics of complex climate mode…
Given a $p$-coin that lands heads with unknown probability $p$, we wish to produce an $f(p)$-coin for a given function $f: (0,1) \rightarrow (0,1)$. This problem is commonly known as the Bernoulli Factory and results on its solvability and complexity have already been obtained. Nevertheless, generic ways to design a practical Bernoulli Factory for a given function $f$ exist only in a few special cases. We present a constructive way to build an efficient Bernoulli Factory when $f(p)$ is a rational function with coefficients in $\mathbb{R}$. Moreover, we extend the Bernoulli Factory problem to a…
In the analysis of Markov chains and processes, it is sometimes convenient to replace an unbounded state space with a "truncated" bounded state space. When such a replacement is made, one often wants to know whether the equilibrium behavior of the truncated chain or process is close to that of the untruncated system. For example, such questions arise naturally when considering numerical methods for computing stationary distributions on unbounded state space. In this paper, we use the principle of "regeneration" to show that the stationary distributions of "fixed state" truncations converge in …
A probabilistic cellular automaton (PCA) can be viewed as a Markov chain. The cells are updated synchronously and independently, according to a distribution depending on a finite neighborhood. We investigate the ergodicity of this Markov chain. A classical cellular automaton is a particular case of PCA. For a 1-dimensional cellular automaton, we prove that ergodicity is equivalent to nilpotency, and is therefore undecidable. We then propose an efficient perfect sampling algorithm for the invariant measure of an ergodic PCA. Our algorithm does not assume any monotonicity property of the local r…
This paper analyzes the performance of sequential importance sampling algorithms for estimating the number of perfect matchings in bipartite graphs. Precise bounds on the number of samples required to yield an accurate estimate are derived. In doing so, moments of permutation statistics are computed using generating functions and nonstandard limit theorems are derived by expressing perfect matchings as a time-inhomogeneous Markov chain.
We study one-sided Markov shifts, corresponding to positively recurrent Markov chains with countable (finite or infinite) state spaces. The following classification problem is considered: when two one-sided Markov shifts are isomorphic up to a measure preserving isomorphism In this paper we solve the problem for the class of rho-uniform (or finitely rho-Bernoulli) one-sided Markov shifts considered in Ru_6 We show that every ergodic rho-uniform Markov shift T can be represented in a canonical form T = T_G by means of a canonical (uniquely determined by T) stochastic graph G. In the canonical f…
