Over the last three decades, there has been a considerable effort within the applied probability community to develop techniques for bounding the convergence rates of general state space Markov chains. Most of these results assume the existence of drift and minorization (d\&m) conditions. It has often been observed that convergence rate bounds based on single-step d\&m tend to be overly conservative, especially in high-dimensional situations. This article builds a framework for studying this phenomenon. It is shown that any convergence rate bound based on a set of d\&m conditions cannot do better than a certain unknown optimal bound. Strategies are designed to put bounds on the optimal bound itself, and this allows one to quantify the extent to which a d\&m-based convergence rate bound can be sharp. The new theory is applied to several examples, including a Gaussian autoregressive process (whose true convergence rate is known), and a Metropolis adjusted Langevin algorithm. The results strongly suggest that convergence rate bounds based on single-step d\&m conditions are quite inadequate in high-dimensional settings.
The "drift-and-minorization" method, introduced and popularized in (Rosenthal, 1995; Meyn and Tweedie, 1994; Meyn and Tweedie, 2012), remains the most popular approach for bounding the convergence rates of Markov chains used in statistical computation. This approach requires estimates of two quantities: the rate at which a single copy of the Markov chain "drifts" towards a fixed "small set", and a "minorization condition" which gives the worst-case time for two Markov chains started within the small set to couple with moderately large probability. In this paper, we build on (Oliveira, 2012; Pe…
This review paper provides an introduction of Markov chains and their convergence rates which is an important and interesting mathematical topic which also has important applications for very widely used Markov chain Monte Carlo (MCMC) algorithm. We first discuss eigenvalue analysis for Markov chains on finite state spaces. Then, using the coupling construction, we prove two quantitative bounds based on minorization condition and drift conditions, and provide descriptive and intuitive examples to showcase how these theorems can be implemented in practice. This paper is meant to provide a gener…
Let $(X_t)$ be a discrete time Markov chain on a general state space. It is well-known that if $(X_t)$ is aperiodic and satisfies a drift and minorization condition, then it converges to its stationary distribution $π$ at an exponential rate. We consider the problem of computing upper bounds for the distance from stationarity in terms of the drift and minorization data. Baxendale showed that these bounds improve significantly if one assumes that $(X_t)$ is reversible with nonnegative eigenvalues (i.e. its transition kernel is a self-adjoint operator on $L^2(π)$ with spectrum contained in $[0,1…
We prove the existence of limiting distributions for a large class of Markov chains on a general state space in a random environment. We assume suitable versions of the standard drift and minorization conditions. In particular, the system dynamics should be contractive on the average with respect to the Lyapunov function and large enough small sets should exist with large enough minorization constants. We also establish that a law of large numbers holds for bounded functionals of the process. Applications to queuing systems, to machine learning algorithms and to autoregressive processes are pr…
This paper considers how to obtain MCMC quantitative convergence bounds which can be translated into tight complexity bounds in high-dimensional {settings}. We propose a modified drift-and-minorization approach, which establishes generalized drift conditions defined in subsets of the state space. The subsets are called the "large sets", and are chosen to rule out some "bad" states which have poor drift property when the dimension of the state space gets large. Using the "large sets" together with a "fitted family of drift functions", a quantitative bound can be obtained which can be translated…
Let X={X_n:n=0,1,2,...} be an irreducible, positive recurrent Markov chain with invariant probability measure π. We show that if X satisfies a one-step minorization condition, then πcan be represented as an infinite mixture. The distributions in the mixture are associated with the hitting times on an accessible atom introduced via the splitting construction of Athreya and Ney [Trans. Amer. Math. Soc. 245 (1978) 493-501] and Nummelin [Z. Wahrsch. Verw. Gebiete 43 (1978) 309-318]. When the small set in the minorization condition is the entire state space, our mixture representation of πreduces t…
This paper studies limit theorems for Markov Chains with general state space under conditions which imply subgeometric ergodicity. We obtain a central limit theorem and moderate deviation principles for additive not necessarily bounded functional of the Markov chains under drift and minorization conditions which are weaker than the Foster-Lyapunov conditions. The regeneration-split chain method and a precise control of the modulated moment of the hitting time to small sets are employed in the proof.
A well-known folklore result in the MCMC community is that the Metropolis-Hastings algorithm mixes quickly for any unimodal target, as long as the tails are not too heavy. Although we've heard this fact stated many times in conversation, we are not aware of any quantitative statement of this result in the literature, and we are not aware of any quick derivation from well-known results. The present paper patches this small gap in the literature, providing a generic bound based on the popular "drift-and-minorization" framework of Rosenthal (1995). Our main contribution is to study two sublevel s…
This paper deals with ergodic theorems for particular time-inhomogeneous Markov processes, whose the time-inhomogeneity is asymptotically periodic. Under a Lyapunov/minorization condition, it is shown that, for any measurable bounded function $f$, the time average $\frac{1}{t} \int_0^t f(X_s)ds$ converges in $\mathbb{L}^2$ towards a limiting distribution, starting from any initial distribution for the process $(X_t)_{t \geq 0}$. This convergence can be improved to an almost sure convergence under an additional assumption on the initial measure. This result will be then applied to show the exis…
We consider an optimal control problem, where a Brownian motion with drift is sequentially observed, and the sign of the drift coefficient changes at jump times of a symmetric two-state Markov process. The Markov process itself is not observable, and the problem consist in finding a {-1,1}-valued process that tracks the unobservable process as close as possible. We present an explicit construction of such a process.
This paper studies the last-column-block-augmented northwest-corner truncation (LC-block-augmented truncation, for short) of discrete-time block-monotone Markov chains under subgeometric drift conditions. The main result of this paper is to present an upper bound for the total variation distance between the stationary probability vectors of a block-monotone Markov chain and its LC-block-augmented truncation. The main result is extended to Markov chains that themselves may not be block monotone but are block-wise dominated by block-monotone Markov chains satisfying modified drift conditions. Fi…
A careful look at rough path topology applied to Brownian motion reveals new possible properties of the well-known Lévy area, in particular the presence of an intrinsic drift of this area. Using renormalization limit of Markov chains on periodic graphs, we present a construction of such a non-trivial drift and give an explicit formula for it. Several examples for which explicit computations are made are included.
This paper studies the augmented truncation of discrete-time block-monotone Markov chains under geometric drift conditions. We first present a bound for the total variation distance between the stationary distributions of an original Markov chain and its augmented truncation. We also obtain such error bounds for more general cases where an original Markov chain itself may not be block-monotone but is block-wise dominated by a block-monotone Markov chain. Finally we discuss the application of our results to GI/G/1-type Markov chains.
We consider a Markov chain on $R^+$ with asymptotically zero drift and finite second moments of jumps which is positive recurrent. A power-like asymptotic behaviour of the invariant tail distribution is proven; such a heavy-tailed invariant measure happens even if the jumps of the chain are bounded. Our analysis is based on test functions technique and on construction of a harmonic function.
The Bouncy Particle Sampler (BPS) is a Monte Carlo Markov Chain algorithm to sample from a target density known up to a multiplicative constant. This method is based on a kinetic piecewise deterministic Markov process for which the target measure is invariant. This paper deals with theoretical properties of BPS. First, we establish geometric ergodicity of the associated semi-group under weaker conditions than in [10] both on the target distribution and the velocity probability distribution. This result is based on a new coupling of the process which gives a quantitative minorization condition …
Many applications in networked control require intermittent access of a controller to a system, as in event-triggered systems or information constrained control applications. Motivated by such applications and extending previous work on Lyapunov-theoretic drift criteria, we establish both subgeometric and geometric rates of convergence for Markov chains under state dependent random time drift criteria. We quantify how the rate of ergodicity, nature of Lyapunov functions, their drift properties, and the distributions of stopping times are related. We finally study an application in networked co…
In this paper, we establish new quantitative convergence bounds for a class of functional autoregressive models in weighted total variation metrics. To derive our results, we show that under mild assumptions, explicit minorization and Foster-Lyapunov drift conditions hold. The main applications and consequences of the bounds we obtain concern the geometric convergence of Euler-Maruyama discretizations of diffusions with identity covariance matrix. Second, as a corollary, we provide a new approach to establish quantitative convergence of these diffusion processes by applying our conclusions in …
In this paper we construct a conservative Markov semi-group with generator $L=Δ+b\cdot\nabla$ on $\mathbb{R}^n$, where $b$ is a divergence-free vector field which belongs to $L^{2}\cap L^{p}$ with $\frac{n}{2}<p$. The research is motivated by the question of understanding the blow-up solutions of the fluid dynamic equations, which attracts a lot of attention in recent years.
Let $(X_n)_{n=0}^\infty$ denote a Markov chain on a Polish space that has a stationary distribution $\varpi$. This article concerns upper bounds on the Wasserstein distance between the distribution of $X_n$ and $\varpi$. In particular, an explicit geometric bound on the distance to stationarity is derived using generalized drift and contraction conditions whose parameters vary across the state space. These new types of drift and contraction allow for sharper convergence bounds than the standard versions, whose parameters are constant. Application of the result is illustrated in the context of …
The goal of this expository paper is to describe conditions which guarantee a central limit theorem for functionals of general state space Markov chains. This is done with a view towards Markov chain Monte Carlo settings and hence the focus is on the connections between drift and mixing conditions and their implications. In particular, we consider three commonly cited central limit theorems and discuss their relationship to classical results for mixing processes. Several motivating examples are given which range from toy one-dimensional settings to complicated settings encountered in Markov ch…
The statement of the mean field approximation theorem in the mean field theory of Markov processes particularly targets the behaviour of population processes with an unbounded number of agents. However, in most real-world engineering applications one faces the problem of analysing middle-sized systems in which the number of agents is bounded. In this paper we build on previous work in this area and introduce the mean drift. We present the concept of population processes and the conditions under which the approximation theorems apply, and then show how the mean drift is derived through a system…
We introduce the cluster index of a multivariate regularly varying stationary sequence and characterize the index in terms of the spectral tail process. This index plays a major role in limit theory for partial sums of regularly varying sequences. We illustrate the use of the cluster index by characterizing infinite variance stable limit distributions and precise large deviation results for sums of multivariate functions acting on a stationary Markov chain under a drift condition.
Let $(X_n)$ be a Markov chain on a standard borelian space $\mathbb{X}$. Any stopping time $τ$ such that $\mathbb{E}_xτ$ is finite for all $x\in\mathbb{X}$ induces a Markov chain in $\mathbb{X}$. In this article, we show that there is a bijection between the invariant measures for the original chain and for the induced one. We then study drift functions and prove a few relations that link the Markov operator for the original chain and for the induced one. The aim is to use this drift function and the induced operator to link the solution to Poisson's equation $(I\_d-P)g=f$ for the original cha…
We consider a linear stochastic differential equation with stochastic drift. We study the problem of approximating the solution of such equation through an Ornstein-Uhlenbeck type process, by using direct methods of calculus of variations. We show that general power cost functionals satisfy the conditions for existence and uniqueness of the approximation. We provide some examples of general interest and we give bounds on the goodness of the corresponding approximations. Finally, we focus on a model of a neuron embedded in a simple network and we study the approximation of its activity, by expl…
We consider a form of state-dependent drift condition for a general Markov chain, whereby the chain subsampled at some deterministic time satisfies a geometric Foster-Lyapunov condition. We present sufficient criteria for such a drift condition to exist, and use these to partially answer a question posed by Connor & Kendall (2007) concerning the existence of so-called 'tame' Markov chains. Furthermore, we show that our 'subsampled drift condition' implies the existence of finite moments for the return time to a small set.
Motivated by applications to proving regularity of solutions to degenerate parabolic equations arising in population genetics, we study existence, uniqueness and the strong Markov property of weak solutions to a class of degenerate stochastic differential equations. The stochastic differential equations considered in our article admit solutions supported in the set $[0,\infty)^n\times\mathbb{R}^m$, and they are degenerate in the sense that the diffusion matrix is not strictly elliptic, as the smallest eigenvalue converges to zero proportional to the distance to the boundary of the domain, and …
We consider a sequence of Markov chains weakly convergent to a diffusion. We suppose that a drift term contains a linearly increasing component. The usual parametrix method fails because of this unbounded drift term. We show how to modify the parametrix method to obtain local limit theorems for this case.
We study the recurrence/transience phase transition for Markov chains on $\mathbb{R}_+$, $\mathbb{R}$, and $\mathbb{R}^2$ whose increments have heavy tails with exponent in $(1,2)$ and asymptotically zero mean. This is the infinite-variance analogue of the classical Lamperti problem. On $\mathbb{R}_+$, for example, we show that if the tail of the positive increments is about $c y^{-α}$ for an exponent $α\in (1,2)$ and if the drift at $x$ is about $b x^{-γ}$, then the critical regime has $γ= α-1$ and recurrence/transience is determined by the sign of $b + cπ\textrm{cosec} (πα)$. On $\mathbb{R}$…
Bacteria are known to exchange genetic information by horizontal gene transfer. Since the frequency of homologous recombination depends on the similarity of recombining segments, several studies examined whether this could lead to the emergence of subspecies. Most of them used simulations of fixed size Wright-Fisher populations, in which the genetic drift should be taken into account. Here, we use non-linear Markov chains to describe a bacterial population evolving under mutation and recombination processes. We consider a population as a space of genomes and a probability measure on it. Thus, …
One-shot coupling is a method of bounding the convergence rate between two copies of a Markov chain in total variation distance, which was first introduced by Roberts and Rosenthal and generalized by Madras and Sezer. The method is divided into two parts: the contraction phase, when the chains converge in expected distance and the coalescing phase, which occurs at the last iteration, when there is an attempt to couple. One-shot coupling does not require the use of any exogenous variables like a drift function or a minorization constant. In this paper, we summarize the one-shot coupling method …
ENS Paris-Saclay（巴黎萨克雷高等师范学院）；Ecole Polytechnique（巴黎综合理工学院）；HSE University（俄罗斯国家研究型高等经济大学）；The Chinese University of Hong Kong（香港中文大学）
This paper studies the exponential stability of random matrix products driven by a general (possibly unbounded) state space Markov chain. It is a cornerstone in the analysis of stochastic algorithms in machine learning (e.g. for parameter tracking in online learning or reinforcement learning). The existing results impose strong conditions such as uniform boundedness of the matrix-valued functions and uniform ergodicity of the Markov chains. Our main contribution is an exponential stability result for the $p$-th moment of random matrix product, provided that (i) the underlying Markov chain sati…
In this paper, we are interested in investigating the perturbation bounds for the stationary distributions for discrete-time or continuous-time Markov chains on a countable state space. For discrete-time Markov chains, two new norm-wise bounds are obtained. The first bound is rather easy to be obtained since the needed condition, equivalent to uniform ergodicity, is imposed on the transition matrix directly. The second bound, which holds for a general (possibly periodic) Markov chain, involves finding a drift function. This drift function is closely related with the mean first hitting times. S…
The existence and uniqueness of the numerical invariant measure of the backward Euler-Maruyama method for stochastic differential equations with Markovian switching is yielded, and it is revealed that the numerical invariant measure converges to the underlying invariant measure in the Wasserstein metric. Under the polynomial growth condition of drift term the convergence rate is estimated. The global Lipschitz condition on the drift coefficients required by Bao et al., 2016 and Yuan et al., 2005 is released. Several examples and numerical experiments are given to verify our theory.
This article gives conditions on a probability measure and drift field b such that for a given killing field k and a given time t > 0, there is function a such that there is a time homogeneous Markov process with infinitesimal generator a((1/2)d^2/dx^2 + b d/dx - k) which meets the given marginal at time t.
In this paper, we establish a probabilistic representation as well as some integration by parts formulae for the marginal law at a given time maturity of some stochastic volatility model with unbounded drift. Relying on a perturbation technique for Markov semigroups, our formulae are based on a simple Markov chain evolving on a random time grid for which we develop a tailor-made Malliavin calculus. Among other applications, an unbiased Monte Carlo path simulation method stems from our formulas so that it can be used in order to numerically compute with optimal complexity option prices as well …
We consider a time-homogeneous Markov chain $X_n$, $n\ge0$, valued in ${\bf R}$. Suppose that this chain is transient, that is, $X_n$ generates a $σ$-finite renewal measure. We prove the key renewal theorem under condition that this chain has asymptotically homogeneous at infinity jumps and asymptotically positive drift.
We present Lyapunov-type conditions for non-strong ergodicity of Markov processes. Some concrete models are discussed including diffusion processes on Riemannian manifolds and Ornstein-Uhlenbeck processes driven by symmetric $α$-stable processes. For SDE driven by $α$-stable process ($α\in (0,2]$) with polynomial drift, the strong ergodicity or not is independent on $α$.
We provide explicit expressions for the constants involved in the characterisation of ergodicity of sub-geometric Markov chains. The constants are determined in terms of those appearing in the assumed drift and one-step minorisation conditions. The result is fundamental for the study of some algorithms where uniform bounds for these constants are needed for a family of Markov kernels. Our result accommodates also some classes of inhomogeneous chains.
In this paper, we give quantitative bounds on the $f$-total variation distance from convergence of an Harris recurrent Markov chain on an arbitrary under drift and minorisation conditions implying ergodicity at a sub-geometric rate. These bounds are then specialized to the stochastically monotone case, covering the case where there is no minimal reachable element. The results are illustrated on two examples from queueing theory and Markov Chain Monte Carlo.
We present a Markov chain example where non-reversibility and an added edge jointly improve mixing time: when a random edge is added to a cycle of $n$ vertices and a Markov chain with a drift is introduced, we get mixing time of $O(n^{3/2})$ with probability bounded away from 0. If only one of the two modifications were performed, the mixing time would stay $Ω(n^2)$.
This paper is a natural continuation of [8], where strong Markov processes are constructed in time inhomogeneous setting with Borel measurable uniformly bounded and uniformly nondegenerate diffusion and drift in $L_{d+1}(\mathbb{R}^{d+1})$. Here we study some properties of these processes such as higher summability of Green's functions, boundedness of resolvent operators in Lebesgue spaces, establish Itô's formula, and so on.
We describe stochastic calculus in the context of processes that are driven by an adapted point process of locally finite intensity and are differentiable between jumps. This includes Markov chains as well as non-Markov processes. By analogy with Itô processes we define the drift and diffusivity, which we then use to describe a general sample path estimate. We then give several examples, including ODE approximation, processes with linear drift, first passage times, and an application to the stochastic logistic model.
We study reflecting Brownian motion with drift constrained to a wedge in the plane. Our first set of results provide necessary and sufficient conditions for existence and uniqueness of a solution to the corresponding submartingale problem with drift, and show that its solution possesses the Markov and Feller properties. Next, we study a version of the problem with absorption at the vertex of the wedge. In this case, we provide a condition for existence and uniqueness of a solution to the problem and some results on the probability of the vertex being reached.
General characterizations of ergodic Markov chains have been developed in considerable detail. In this paper, we study the transience for discrete-time Markov chains on general state spaces, including the geometric transience and algebraic transience. Criteria are presented through establishing the drift condition and considering the first return time. As an application, we give explicit criteria for the random walk on the half line and the skip-free chain on nonnegative integers.
We consider a time inhomogeneous strong Markov process $(ξ_t)_{t\ge 0}$ taking values in a Polish state space whose semigroup has a $T$-periodic structure. We give simple conditions which imply ergodicity of the grid chain $(ξ_{kT})_{k\in \mathbb{N}_0}$ In case of $d$-dimensional possibly degenerate diffusions, the conditions are stated in terms of drift and diffusion coefficient of the process.
A family of Feller branching diffusions $Z^x$, $x \ge 0$, with nonlinear drift and initial value $x$ can, with a suitable coupling over the {\em ancestral masses} $x$, be viewed as a path-valued process indexed by $x$. For a coupling due to Dawson and Li, which in case of a linear drift describes the corresponding Feller branching diffusion, and in our case makes the path-valued process Markovian, we find an SDE solved by $Z$, which is driven by a random point measure on excursion space. In this way we are able to identify the infinitesimal generator of the path-valued process. We also establi…
We construct an irreversible local Markov dynamics on configurations of up-right paths on a discrete two-dimensional torus, that preserves the Gibbs measures for the six vertex model. An additional feature of the dynamics is a conjecturally nontrivial drift of the height function.
