We estimate the distance in total variation between the law of a finite state Markov process at time t, starting from a given initial measure, and its unique invariant measure. We derive upper bounds for the time to reach the equilibrium. As an example of application we consider a special case of finite state Markov process in random environment: the Metropolis dynamics of the Random Energy Model. We also study the process of the environment as seen from the process.
We prove that every Markov solution to the three dimensional Navier-Stokes equation with periodic boundary conditions driven by additive Gaussian noise is uniquely ergodic. The convergence to the (unique) invariant measure is exponentially fast. Moreover, we give a well-posedness criterion for the equations in terms of invariant measures. We also analyse the energy balance and identify the term which ensures equality in the balance.
In this paper, we study the mean field limit of interacting particles with memory that are governed by a system of interacting non-Markovian Langevin equations. Under the assumption of quasi-Markovianity (i.e. that the memory in the system can be described using a finite number of auxiliary processes), we pass to the mean field limit to obtain the corresponding McKean-Vlasov equation in an extended phase space. We obtain the fundamental solution (Green's function) for this equation, for the case of a quadratic confining potential and a quadratic (Curie-Weiss) interaction. Furthermore, for nonc…
Improved rates of convergence for ergodic Markov chains and relaxed conditions for them, as well as analogous convergence results for some non-homogeneous Markov chains are studied. The setting from the previous works is extended. Examples are provided where the new bounds are better and where they give the same convergence rate as in the classical Markov -- Dobrushin inequality.
Necessary and sufficient conditions for convergence in distribution of first-rare-event times and convergence in Skorokhod J-topology of first-rare-event-time processes for perturbed semi-Markov processes with finite phase space are obtained.
We study Markovian symmetry and non-symmetry random evolutions in $\mathbf{R}^n$. Weak convergence of Markovian symmetry random evolution to Wiener process and of Markovian non-symmetry random evolution to a diffusion process with drift is proved using problems of singular perturbation for the generators of evolutions. Relative compactness in $\mathbf{D}_{\mathbf{R}^n\timesΘ}[0,\infty)$ of the families of Markovian random evolutions is also shown.
We study inhomogeneous continuous-time weakly ergodic Markov chains with a finite state space. We introduce the notion of a Markov chain with the regular structure of an infinitesimal matrix and study the sharp upper bounds on the rate of convergence for such class of Markov chains.
We study a class of strongly irreducible, multidimensional, topological Markov shifts, comparing two notions of "symmetric measure": exchangeability and the Gibbs (or conformal) property. We show that equilibrium measures for such shifts (unique and weak Bernoulli in the one dimensional case) exhibit a variety of spectral properties.
Mean convergence of Markovian spherical averages is established for a measure-preserving action of a finitely-generated free group on a probability space. We endow the set of generators with a generalized Markov chain and establish the mean convergence of resulting spherical averages in this case under mild nondegeneracy assumptions on the stochastic matrix $Π$ defining our Markov chain. Equivalently, we establish the triviality of the tail sigma-algebra of the corresponding Markov operator. This convergence was previously known only for symmetric Markov chains, while the conditions ensuring c…
The exponential rate of convergence for some Markov operators is established. The operators correspond to continuous iterated function systems which are a very useful tool in some cell cycle models.
This paper is devoted to solving a time-inconsistent risk-sensitive control problem with parameter $\e$ and its limit case ($\e\rightarrow0^+$) for countable-stated Markov decision processes (MDPs for short). Since the cost functional is time-inconsistent, it is impossible to find a global optimal strategy for both cases. Instead, for each case, we will prove the existence of time-inconstant equilibrium strategies which verify the so-called step-optimality. Moreover, we prove the convergence of $\e$-equilibriums and the corresponding value functions as $\e\rightarrow0^+$.
For each $n$ let $Y^n_t$ be a continuous time symmetric Markov chain with state space $n^{-1} \Z^d$. A condition in terms of the conductances is given for the convergence of the $Y^n_t$ to a symmetric Markov process $Y_t$ on $\R^d$. We have weak convergence of $\{Y^n_t: t\leq t_0\}$ for every $t_0$ and every starting point. The limit process $Y$ has a continuous part and may also have jumps.
Improved rates of convergence for ergodic homogeneous Markov chains are studied. In comparison to the earlier papers the setting is also generalised to the case without a unique dominated measure. Examples are provided where the new bound is compared with the classical Markov -- Dobrushin inequality and with the second eigenvalue of the transition matrix for finite state spaces.
We consider continuous-time Markov chains which display a family of wells at the same depth. We provide sufficient conditions which entail the convergence of the finite-dimensional distributions of the order parameter to the ones of a finite state Markov chain. We also show that the state of the process can be represented as a time-dependent convex combination of metastable states, each of which is supported on one well.
In this paper, we give quantitative bounds on the $f$-total variation distance from convergence of an Harris recurrent Markov chain on an arbitrary under drift and minorisation conditions implying ergodicity at a sub-geometric rate. These bounds are then specialized to the stochastically monotone case, covering the case where there is no minimal reachable element. The results are illustrated on two examples from queueing theory and Markov Chain Monte Carlo.
We suggest an approach to obtaining general two-sided bounds on the rate of convergence in terms of special "weighted" norms related to total variation. Some important classes of continuous-time Markov chains are considered: birth-death-catastrophes processes, queueing models with batch arrivals and group services, chains with absorption in zero.
This paper focuses on a class of continuous-time controlled Markov chains with time-inconsistent and distribution-dependent cost functional (in some appropriate sense). A new definition of time-inconsistent distribution-dependent equilibrium in closed-loop sense is given and its existence and uniqueness have been established. Because of the time-inconsistency, it is proved that the equilibrium is locally optimal in an appropriate sense. Moreover, it has been shown that our problem is essentially equivalent to an infinite-player mean-field game with time-inconsistent cost.
In the paper, we study a new rate of convergence estimate for homogeneous discrete-time nonlinear Markov chains based on the Markov-Dobrushin condition. This result generalizes the convergence estimates for any positive number of transition steps. An example of a class such a process provided indicates that such types of estimates considering several transition steps may be applicable when one transition can not guarantee any convergence. Moreover, a better estimate can be obtained for a higher number of transitions steps. A law of large numbers is presented for a class of ergodic nonlinear Ma…
We provide a abstract framework to prove total variation convergence result with arbitrary rate for numerical scheme for SDE. In particular we show that under standard weak approximation properties of scheme such as Euler we can obtain total variation convergence with any desired rate by building a specific approximation for the SDE.
Studying the subexponential convergence towards equilibrium of a strong Markov process, we exhibit an intermediate Lyapunov condition equivalent to the control of some moment of a hitting time. This provides a link, similar (although more intricate) to the one existing in the exponential case, between the coupling method and the approach based on the existence of a Lyapunov function for the generator, in the context of the subexponential rates found by Fort-Roberts (2005), Douc-Fort-Guillin (2009) and Hairer (2016).
In this note, a general approach to the study of non-stationary Markov chains with catastrophes and the corresponding queuing models is considered, as well as to obtain estimates of the limiting regime itself. As an illustration, an example of a queuing model is studied.
Our aim is to find sufficient conditions for weak convergence of stochastic integrals with respect to the state occupation measure of a Markov chain. First, we study properties of the state indicator function and the state occupation measure of a Markov chain. In particular, we establish weak convergence of the state occupation measure under a scaling of the generator matrix. Then, relying on the connection between the state occupation measure and the Dynkin martingale related to the state indicator function, we provide sufficient conditions for weak convergence of stochastic integrals with re…
This paper deals with three major types of convergence of probability measures on metric spaces: weak convergence, setwise converges, and convergence in the total variation. First, it describes and compares necessary and sufficient conditions for these types of convergence, some of which are well-known, in terms of convergence of probabilities of open and closed sets and, for the probabilities on the real line, in terms of convergence of distribution functions. Second, it provides % convenient criteria for weak and setwise convergence of probability measures and continuity of stochastic kernel…
We study the rate of weak convergence of Markov chains to diffusion processes under suitable but quite general assumptions. We give an example in the financial framework, applying the convergence analysis to a multiple jumps tree approximation of the CIR process. Then, we combine the Markov chain approach with other numerical techniques in order to handle the different components in jump-diffusion coupled models. We study the speed of convergence of this hybrid approach and we provide an example in finance, applying our results to a tree-finite difference approximation in the Heston or Bates m…
A nonnegative coarse Ricci curvature for a Markov chain and the existence of an attractive point implies the concentration of the invariant probability measure around this point. The mass outside balls centered at the attractive point, as a function of the radius, decreases at least as fast as the exponential of a double integral of the coarse Ricci curvature. This is exactly the behaviour of the density of the reversible measure for diffusion processes on the real line.
In this note, we are concerned with the subgeometric rate of convergence of a Markov chain with discrete time parameter to its invariant measure in the $f$-norm. We clarify how three typical subgeometric rates of convergence are inherited under a discrete time version of Bochner's subordination. The crucial point is to establish the corresponding moment estimates for discrete time subordinators under some reasonable conditions on the underlying Bernstein function.
The paper is concerned with the equilibrium distributions of continuous-time density dependent Markov processes on the integers. These distributions are known typically to be approximately normal, and the approximation error, as measured in Kolmogorov distance, is of the smallest order that is compatible with their having integer support. Here, an approximation in the much stronger total variation norm is established, without any loss in the asymptotic order of accuracy; the approximating distribution is a translated Poisson distribution having the same variance and (almost) the same mean. Our…
Generalization of the Lorden's inequality is an excellent tool for obtaining strong upper bounds for the convergence rate for various complicated stochastic models. This paper demonstrates a method for obtaining such bounds for some generalization of the Markov modulated Poisson process (MMPP). The proposed method can be applied to reliability and queuing theory.
In this brief paper we find computable exponential convergence rates for a large class of stochastically ordered Markov processes. We extend the result of Lund, Meyn, and Tweedie (1996), who found exponential convergence rates for stochastically ordered Markov processes starting from a fixed initial state, by allowing for a random initial condition that is also stochastically ordered. Our bounds are formulated in terms of moment-generating functions of hitting times. To illustrate our result, we find an explicit exponential convergence rate for an M/M/1 queue beginning in equilibrium and then …
We provide explicit expressions for the constants involved in the characterisation of ergodicity of sub-geometric Markov chains. The constants are determined in terms of those appearing in the assumed drift and one-step minorisation conditions. The result is fundamental for the study of some algorithms where uniform bounds for these constants are needed for a family of Markov kernels. Our result accommodates also some classes of inhomogeneous chains.
We study the quasi-stationary behavior of multidimensional processes absorbed when one of the coordinates vanishes. Our results cover competitive or weakly cooperative Lotka-Volterra birth and death processes and Feller diffusions with competitive Lotka-Volterra interaction. To this aim, we develop original non-linear Lyapunov criteria involving two Lyapunov functions, which apply to general Markov processes.
In Latouche and Nguyen (2015), the authors constructed a sequence of stochastic fluid processes and showed that it converges weakly to a Markov-modulated Brownian motion (MMBM). Here, we construct a different sequence of stochastic fluid processes and show that it converges strongly to an MMBM. To the best of our knowledge, this is the first result on strong convergence to a Markov-modulated Brownian motion. We also prove that the rate of this almost sure convergence is $o(n^{-1/2} \log n)$. When reduced to the special case of standard Brownian motion, our convergence rate is an improvement ov…
We determine the convergence speed of a numerical scheme for approximating one-dimensional continuous strong Markov processes. The scheme is based on the construction of coin tossing Markov chains whose laws can be embedded into the process with a sequence of stopping times. Under a mild condition on the process' speed measure we prove that the approximating Markov chains converge at fixed times at the rate of $1/4$ with respect to every $p$-th Wasserstein distance. For the convergence of paths, we prove any rate strictly smaller than $1/4$. Our results apply, in particular, to processes with …
We provide necessary and sufficient conditions for convergence of exponential integrals of Markov additive processes. Other than in the classical Lévy case studied by Erickson and Maller we have to distinguish between almost sure convergence and convergence in probability. Our proofs rely on recent results on perpetuities in a Markovian environment by Alsmeyer and Buckmann.
Using elementary methods, we prove that for a countable Markov chain $P$ of ergodic degree $d > 0$ the rate of convergence towards the stationary distribution is subgeometric of order $n^{-d}$, provided the initial distribution satisfies certain conditions of asymptotic decay. An example, modelling a renewal process and providing a markovian approximation scheme in dynamical system theory, is worked out in detail, illustrating the relationships between convergence behaviour, analytic properties of the generating functions associated to transition probabilities and spectral properties of the Ma…
We provide an original and general sufficient criterion ensuring the exponential contraction of Feynman-Kac semi-groups of penalized processes. This criterion is applied to time-inhomogeneous one-dimensional diffusion processes conditioned not to hit 0 and to penalized birth and death processes evolving in a quenched random environment.
The paper studies an improved estimate for the rate of convergence for nonlinear homogeneous discrete-time Markov chains. These processes are nonlinear in terms of the distribution law. Hence, the transition kernels are dependent on the current probability distributions of the process apart from being dependent on the current state. Such processes often act as limits for large-scale systems of dependent Markov chains with interaction. The paper generalizes the convergence results by taking the estimate over two steps. Such an approach keeps the existence and uniqueness results under assumption…
In this paper we consider the convergence of the conditional entropy to the entropy rate for Markov chains. Convergence of certain statistics of long range dependent processes, such as the sample mean, is slow. It has been shown in Carpio and Daley \cite{carpio2007long} that the convergence of the $n$-step transition probabilities to the stationary distribution is slow, without quantifying the convergence rate. We prove that the slow convergence also applies to convergence to an information-theoretic measure, the entropy rate, by showing that the convergence rate is equivalent to the convergen…
This paper considers an approximation usually used when implementing Ramaswami's recursion for the stationary distribution of the M/G/1-type Markov chain. The approximation is called the level-increment-truncation approximation because it truncates level increment at a given threshold. The main contribution of this paper is to present a geometric convergence formula of the level-wise difference between the respective stationary distributions of the original M/G/1-type Markov chain and its LI truncation approximation under the assumption that the level-increment distribution is light-tailed.
We analyse the structure of imprecise Markov chains and study their convergence by means of accessibility relations. We first identify the sets of states, so-called minimal permanent classes, that are the minimal sets capable of containing and preserving the whole probability mass of the chain. These classes generalise the essential classes known from the classical theory. We then define a class of extremal imprecise invariant distributions and show that they are uniquely determined by the values of the upper probability on minimal permanent classes. Moreover, we give conditions for unique con…
We are interested in the rate of convergence of a subordinate Markov process to its invariant measure. Given a subordinator and the corresponding Bernstein function (Laplace exponent) we characterize the convergence rate of the subordinate Markov process; the key ingredients are the rate of convergence of the original process and the (inverse of the) Bernstein function. At a technical level, the crucial point is to bound three types of moments (sub-exponential, algebraic and logarithmic) for subordinators as time $t$ tends to infinity. At the end we discuss some concrete models and we show tha…
In this paper, we provide sufficient conditions for the existence of the invariant distribution and for subgeometric rates of convergence in Wasserstein distance for general state-space Markov chains which are (possibly) not irreducible. Compared to previous work, our approach is based on a purely probabilistic coupling construction which allows to retrieve rates of convergence matching those previously reported for convergence in total variation. Our results are applied to establish the subgeometric ergodicity in Wasserstein distance of non-linear autoregressive models and of the pre-conditio…
Let $\{X_n\}_{n\in\N}$ be a Markov chain on a measurable space $\X$ with transition kernel $P$ and let $V:\X\r[1,+\infty)$. The Markov kernel $P$ is here considered as a linear bounded operator on the weighted-supremum space $\cB_V$ associated with $V$. Then the combination of quasi-compactness arguments with precise analysis of eigen-elements of $P$ allows us to estimate the geometric rate of convergence $ρ_V(P)$ of $\{X_n\}_{n\in\N}$ to its invariant probability measure in operator norm on $\cB_V$. A general procedure to compute $ρ_V(P)$ for discrete Markov random walks with identically dist…
Nonlinear Markov Chains (nMC) are regarded as the original (linear) Markov Chains with nonlinear small perturbations. It fits real-world data better, but its associated properties are difficult to describe. A new approach is proposed to analyze the ergodicity and even estimate the convergence bounds of nMC, which is more precise than existing results. In the new method, Coupling Markov about homogeneous Markov chains is applied to reconstitute the relationship between distribution at any times and the limiting distribution. The convergence bounds can be provided by the transition probability m…
We study the convergence rate to stationarity for a class of exchangeable partition-valued Markov chains called cut-and-paste chains. The law governing the transitions of a cut-and-paste chain are determined by products of i.i.d. stochastic matrices, which describe the chain induced on the simplex by taking asymptotic frequencies. Using this representation, we establish upper bounds for the mixing times of ergodic cut-and-paste chains, and under certain conditions on the distribution of the governing random matrices we show that the "cutoff phenomenon" holds.
Consider a filtering process associated to a hidden Markov model with densities for which both the state space and the observation space are complete, separable, metric spaces. If the underlying, hidden Markov chain is strongly ergodic and the filtering process fulfills a certain coupling condition we prove that, in the limit, the distribution of the filtering process is independent of the initial distribution of the hidden Markov chain. If furthermore the hidden Markov chain is uniformly ergodic, then we prove that the filtering process converges in distribution.
