This paper studies limit theorems for Markov Chains with general state space under conditions which imply subgeometric ergodicity. We obtain a central limit theorem and moderate deviation principles for additive not necessarily bounded functional of the Markov chains under drift and minorization conditions which are weaker than the Foster-Lyapunov conditions. The regeneration-split chain method and a precise control of the modulated moment of the hitting time to small sets are employed in the proof.
In this note, we are concerned with the subgeometric rate of convergence of a Markov chain with discrete time parameter to its invariant measure in the $f$-norm. We clarify how three typical subgeometric rates of convergence are inherited under a discrete time version of Bochner's subordination. The crucial point is to establish the corresponding moment estimates for discrete time subordinators under some reasonable conditions on the underlying Bernstein function.
In this paper, we give quantitative bounds on the $f$-total variation distance from convergence of an Harris recurrent Markov chain on an arbitrary under drift and minorisation conditions implying ergodicity at a sub-geometric rate. These bounds are then specialized to the stochastically monotone case, covering the case where there is no minimal reachable element. The results are illustrated on two examples from queueing theory and Markov Chain Monte Carlo.
This paper studies the last-column-block-augmented northwest-corner truncation (LC-block-augmented truncation, for short) of discrete-time block-monotone Markov chains under subgeometric drift conditions. The main result of this paper is to present an upper bound for the total variation distance between the stationary probability vectors of a block-monotone Markov chain and its LC-block-augmented truncation. The main result is extended to Markov chains that themselves may not be block monotone but are block-wise dominated by block-monotone Markov chains satisfying modified drift conditions. Fi…
We derive sufficient conditions for subgeometric f-ergodicity of strongly Markovian processes. We first propose a criterion based on modulated moment of some delayed return-time to a petite set. We then formulate a criterion for polynomial f-ergodicity in terms of a drift condition on the generator. Applications to specific processes are considered, including Langevin tempered diffusions on R^n and storage models.
We establish subgeometric bounds on convergence rate of general Markov processes in the Wasserstein metric. In the discrete time setting we prove that the Lyapunov drift condition and the existence of a "good" $d$-small set imply subgeometric convergence to the invariant measure. In the continuous time setting we obtain the same convergence rate provided that there exists a "good" $d$-small set and the Douc-Fort-Guillin supermartingale condition holds. As an application of our results, we prove that the Veretennikov-Khasminskii condition is sufficient for subexponential convergence of strong s…
We consider a form of state-dependent drift condition for a general Markov chain, whereby the chain subsampled at some deterministic time satisfies a geometric Foster-Lyapunov condition. We present sufficient criteria for such a drift condition to exist, and use these to partially answer a question posed by Connor & Kendall (2007) concerning the existence of so-called 'tame' Markov chains. Furthermore, we show that our 'subsampled drift condition' implies the existence of finite moments for the return time to a small set.
This paper studies the subgeometric convergence of the stationary distribution in taking the infinite-level limit of a finite-level M/G/1-type Markov chain, that is, in letting the upper boundary level go to infinity. This study is performed through the fundamental deviation matrix, which is a block-decomposition-friendly solution for the Poisson equation of the deviation matrix. The fundamental deviation matrix yields a difference formula for the respective stationary distributions of the finite-level chain and the corresponding infinite-level chain. The difference formula plays a crucial rol…
This paper considers the level-increment (LI) truncation approximation of M/G/1-type Markov chains. The LI truncation approximation is useful for implementing the M/G/1 paradigm, which is the framework for computing the stationary distribution of M/G/1-type Markov chains. The main result of this paper is a subgeometric convergence formula for the total variation distance between the original stationary distribution and its LI truncation approximation. Suppose that the equilibrium level-increment distribution is subexponential, and that the downward transition matrix is rank one. We then show t…
We are interested in the rate of convergence of a subordinate Markov process to its invariant measure. Given a subordinator and the corresponding Bernstein function (Laplace exponent) we characterize the convergence rate of the subordinate Markov process; the key ingredients are the rate of convergence of the original process and the (inverse of the) Bernstein function. At a technical level, the crucial point is to bound three types of moments (sub-exponential, algebraic and logarithmic) for subordinators as time $t$ tends to infinity. At the end we discuss some concrete models and we show tha…
In this paper, we provide sufficient conditions for the existence of the invariant distribution and for subgeometric rates of convergence in Wasserstein distance for general state-space Markov chains which are (possibly) not irreducible. Compared to previous work, our approach is based on a purely probabilistic coupling construction which allows to retrieve rates of convergence matching those previously reported for convergence in total variation. Our results are applied to establish the subgeometric ergodicity in Wasserstein distance of non-linear autoregressive models and of the pre-conditio…
We extend the hypocoercivity framework for piecewise-deterministic Markov process (PDMP) Monte Carlo established in [Andrieu et. al. (2018)] to heavy-tailed target distributions, which exhibit subgeometric rates of convergence to equilibrium. We make use of weak Poincaré inequalities, as developed in the work of [Grothaus and Wang (2019)], the ideas of which we adapt to the PDMPs of interest. On the way we report largely potential-independent approaches to bounding explicitly solutions of the Poisson equation of the Langevin diffusion and its first and second derivatives, required here to cont…
It is well known that stationary geometrically ergodic Markov chains are $β$-mixing (absolutely regular) with geometrically decaying mixing coefficients. Furthermore, for initial distributions other than the stationary one, geometric ergodicity implies $β$-mixing under suitable moment assumptions. In this note we show that similar results hold also for subgeometrically ergodic Markov chains. In particular, for both stationary and other initial distributions, subgeometric ergodicity implies $β$-mixing with subgeometrically decaying mixing coefficients. Although this result is simple it should p…
This paper deals with the ergodicity and the existence of a strong law of large numbers for adaptive Markov Chain Monte Carlo. We show that a diminishing adaptation assumption together with a drift condition for positive recurrence is enough to imply ergodicity. Strengthening the drift condition to a polynomial drift condition yields a strong law of large numbers for possibly unbounded functions. These results broaden considerably the class of adaptive MCMC algorithms for which rigorous analysis is now possible. As an example, we give a detailed analysis of the Adaptive Metropolis Algorithm of…
We prove a central limit theorem for a general class of adaptive Markov Chain Monte Carlo algorithms driven by sub-geometrically ergodic Markov kernels. We discuss in detail the special case of stochastic approximation. We use the result to analyze the asymptotic behavior of an adaptive version of the Metropolis Adjusted Langevin algorithm with a heavy tailed target density.
We study a class of Markov processes that combine local dynamics, arising from a fixed Markov process, with regenerations arising at a state-dependent rate. We give conditions under which such processes possess a given target distribution as their invariant measures, thus making them amenable for use within Monte Carlo methodologies. Since the regeneration mechanism can compensate the choice of local dynamics, while retaining the same invariant distribution, great flexibility can be achieved in selecting local dynamics, and the mathematical analysis is simplified. We give straightforward condi…
For a Harris ergodic Markov chain $(X_n)_{n\ge 0}$, on a general state space, started from the so called small measure or from the stationary distribution we provide optimal estimates for Orlicz norms of sums $\sum_{i=0}^τf(X_i)$, where $τ$ is the first regeneration time of the chain. The estimates are expressed in terms of other Orlicz norms of the function $f$ (wrt the stationary distribution) and the regeneration time $τ$ (wrt the small measure). We provide applications to tail estimates for additive functionals of the chain $(X_n)$ generated by unbounded functions as well as to classical l…
Many applications in networked control require intermittent access of a controller to a system, as in event-triggered systems or information constrained control applications. Motivated by such applications and extending previous work on Lyapunov-theoretic drift criteria, we establish both subgeometric and geometric rates of convergence for Markov chains under state dependent random time drift criteria. We quantify how the rate of ergodicity, nature of Lyapunov functions, their drift properties, and the distributions of stopping times are related. We finally study an application in networked co…
Using elementary methods, we prove that for a countable Markov chain $P$ of ergodic degree $d > 0$ the rate of convergence towards the stationary distribution is subgeometric of order $n^{-d}$, provided the initial distribution satisfies certain conditions of asymptotic decay. An example, modelling a renewal process and providing a markovian approximation scheme in dynamical system theory, is worked out in detail, illustrating the relationships between convergence behaviour, analytic properties of the generating functions associated to transition probabilities and spectral properties of the Ma…
We study a model of a polling system, that is, a collection of $d$ queues with a single server that switches from queue to queue. The service time distribution and arrival rates change randomly every time a queue is emptied. This model is mapped to a mathematically equivalent model of a random walk with random choice of transition probabilities, a model which is of independent interest. All our results are obtained using methods from the constructive theory of Markov chains. We determine conditions for the existence of polynomial moments of hitting times for the random walk. An unusual phenome…
We consider a model, introduced by Boldrighini, Minlos and Pellegrinotti, of random walks in dynamical random environments on the integer lattice Z^d with d>=1. In this model, the environment changes over time in a Markovian manner, independently across sites, while the walker uses the environment at its current location in order to make the next transition. In contrast with the cluster expansions approach of Boldrighini, Minlos and Pellegrinotti, we follow a probabilistic argument based on regeneration times. We prove an annealed SLLN and invariance principle for any dimension, and provide a …
Consider a strong Markov process in continuous time, taking values in some Polish state space. Recently, Douc, Fort and Guillin (2009) introduced verifiable conditions in terms of a supermartingale property implying an explicit control of modulated moments of hitting times. We show how this control can be translated into a control of polynomial moments of abstract regeneration times which are obtained by using the regeneration method of Nummelin, extended to the time-continuous context. As a consequence, if a $p-$th moment of the regeneration times exists, we obtain non asymptotic deviation bo…
Let $X$ be a Harris recurrent strong Markov process in continuous time with general Polish state space $E,$ having invariant measure $μ.$ In this paper we use the regeneration method to derive non asymptotic deviation bounds for $$P_{x} (|\int_0^tf(X_s)ds|\geq t^{\frac12 + η} \ge)$$ in the positive recurrent case, for nice functions $f$ with $μ(f) =0 $ ($f$ must be a charge). We generalize these bounds to the fully null-recurrent case in the moderate deviations regime. We obtain a Gaussian contentration bound for all functions $f$ which are a charge. The rate of convergence is expressed in ter…
This paper surveys various results about Markov chains on general (non-countable) state spaces. It begins with an introduction to Markov chain Monte Carlo (MCMC) algorithms, which provide the motivation and context for the theory which follows. Then, sufficient conditions for geometric and uniform ergodicity are presented, along with quantitative bounds on the rate of convergence to stationarity. Many of these results are proved using direct coupling constructions based on minorisation and drift conditions. Necessary and sufficient conditions for Central Limit Theorems (CLTs) are also presente…
We introduce Path-ZVA: an efficient simulation technique for estimating the probability of reaching a rare goal state before a regeneration state in a (discrete-time) Markov chain. Standard Monte Carlo simulation techniques do not work well for rare events, so we use importance sampling; i.e., we change the probability measure governing the Markov chain such that transitions `towards' the goal state become more likely. To do this we need an idea of distance to the goal state, so some level of knowledge of the Markov chain is required. In this paper, we use graph analysis to obtain this knowled…
A rigorous and largely self-contained account of (a) the bread-and-butter concepts and techniques in Markov chain theory and (b) the long-term behaviour of chains. As much as possible, the treatment is probabilistic instead of analytical (I stay away from semigroup theory). Personally, I tend to find that the intuition lies with the former and not the latter. This manuscript is geared towards those interested in the use of Markov chains as models of real-life phenomena. For this reason, I focus on the type of chains most commonly encountered in practice (time-homogeneous, minimal, and right-co…
We derive an invariance principle for the lift to the rough path topology of stochastic processes with delayed regenerative increments under an optimal moment condition. An interesting feature of the result is the emergence of area anomaly, a correction term in the second level of the limiting rough path which is identified as the average stochastic area on a regeneration interval. A few applications include random walks in random environment and additive functionals of recurrent Markov chains. The result is formulated in the p-variation settings, where a rough Donsker Theorem is available und…
We consider the first exit time of a nonnegative Harris-recurrent Markov process from the interval $[0,A]$ as $A\to\infty$. We provide an alternative method of proof of asymptotic exponentiality of the first exit time (suitably standardized) that does not rely on embedding in a regeneration process. We show that under certain conditions the moment generating function of a suitably standardized version of the first exit time converges to that of $\Exp(1)$, and we connect between the standardizing constant and the quasi-stationary distribution (assuming it exists). The results are applied to the…
In the analysis of Markov chains and processes, it is sometimes convenient to replace an unbounded state space with a "truncated" bounded state space. When such a replacement is made, one often wants to know whether the equilibrium behavior of the truncated chain or process is close to that of the untruncated system. For example, such questions arise naturally when considering numerical methods for computing stationary distributions on unbounded state space. In this paper, we use the principle of "regeneration" to show that the stationary distributions of "fixed state" truncations converge in …
We consider random walks in dynamic random environments, with an environment generated by the time-reversal of a Markov process from the oriented percolation universality class. If the influence of the random medium on the walk is small in space-time regions where the medium is typical, we obtain a law of large numbers and an averaged central limit theorem for the walk via a regeneration construction under suitable coarse-graining. Such random walks occur naturally as spatial embeddings of ancestral lineages in spatial population models with local regulation. We verify that our assumptions hol…
