Using Lamperti's relationship between Lévy processes and positive self-similar Markov processes (pssMp), we study the weak convergence of the law $\mathbb{P}_x$ of a pssMp starting at $x>0$, in the Skorohod space of càdlàg paths, when $x$ tends to 0. To do so, we first give conditions which allow us to construct a càdlàg Markov process $X^{(0)}$, starting from 0, which stays positive and verifies the scaling property. Then we establish necessary and sufficient conditions for the laws $\mathbb{P}_x$ to converge weakly to the law of $X^{(0)}$ as $x$ goes to 0. In particular, this answers a question raised by Lamperti [Z. Wahrsch. Verw. Gebiete 22 (1972) 205--225] about the Feller property for pssMp at $x=0$.
We develop criteria for recurrence and transience of one-dimensional Markov processes which have jumps and oscillate between $+\infty$ and $-\infty$. The conditions are based on a Markov chain which only consists of jumps (overshoots) of the process into complementary parts of the state space. In particular we show that a stable-like process with generator $-(-Δ)^{α(x)/2}$ such that $α(x)=α$ for $x<-R$ and $α(x)=β$ for $x>R$ for some $R>0$ and $α,β\in(0,2)$ is transient if and only if $α+β<2$, otherwise it is recurrent. As a special case this yields a new proof for the recurrence, point recurr…
We prove that for a random walk on the real line whose increments have zero mean and are either integer-valued or spread out (i.e. the distributions of the steps of the walk are eventually non-singular), the Markov chain of overshoots above a fixed level converges in total variation to its stationary distribution. We find the explicit form of this distribution heuristically and then prove its invariance using a time-reversal argument. If, in addition, the increments of the walk are in the domain of attraction of a non-one-sided $α$-stable law with index $α\in(1,2)$ (resp. have finite variance)…
For a stopped diffusion process in a multidimensional time-dependent domain $\D$, we propose and analyse a new procedure consisting in simulating the process with an Euler scheme with step size $Δ$ and stopping it at discrete times $(iΔ)_{i\in\N^*}$ in a modified domain, whose boundary has been appropriately shifted. The shift is locally in the direction of the inward normal $n(t,x)$ at any point $(t,x)$ on the parabolic boundary of $\D$, and its amplitude is equal to $0.5826 (...) |n^*σ|(t,x)\sqrt Δ$ where $σ$ stands for the diffusion coefficient of the process. The procedure is thus extremel…
Understanding the space-time features of how a Lévy process crosses a constant barrier for the first time, and indeed the last time, is a problem which is central to many models in applied probability such as queueing theory, financial and actuarial mathematics, optimal stopping problems, the theory of branching processes to name but a few. In \cite{KD} a new quintuple law was established for a general Lévy process at first passage below a fixed level. In this article we use the quintuple law to establish a family of related joint laws, which we call $n$-tuple laws, for Lévy processes, Lévy pr…
Improved rates of convergence for ergodic Markov chains and relaxed conditions for them, as well as analogous convergence results for some non-homogeneous Markov chains are studied. The setting from the previous works is extended. Examples are provided where the new bounds are better and where they give the same convergence rate as in the classical Markov -- Dobrushin inequality.
Necessary and sufficient conditions for convergence in distribution of first-rare-event times and convergence in Skorokhod J-topology of first-rare-event-time processes for perturbed semi-Markov processes with finite phase space are obtained.
We study Markovian symmetry and non-symmetry random evolutions in $\mathbf{R}^n$. Weak convergence of Markovian symmetry random evolution to Wiener process and of Markovian non-symmetry random evolution to a diffusion process with drift is proved using problems of singular perturbation for the generators of evolutions. Relative compactness in $\mathbf{D}_{\mathbf{R}^n\timesΘ}[0,\infty)$ of the families of Markovian random evolutions is also shown.
We study inhomogeneous continuous-time weakly ergodic Markov chains with a finite state space. We introduce the notion of a Markov chain with the regular structure of an infinitesimal matrix and study the sharp upper bounds on the rate of convergence for such class of Markov chains.
The exponential rate of convergence for some Markov operators is established. The operators correspond to continuous iterated function systems which are a very useful tool in some cell cycle models.
For each $n$ let $Y^n_t$ be a continuous time symmetric Markov chain with state space $n^{-1} \Z^d$. A condition in terms of the conductances is given for the convergence of the $Y^n_t$ to a symmetric Markov process $Y_t$ on $\R^d$. We have weak convergence of $\{Y^n_t: t\leq t_0\}$ for every $t_0$ and every starting point. The limit process $Y$ has a continuous part and may also have jumps.
Improved rates of convergence for ergodic homogeneous Markov chains are studied. In comparison to the earlier papers the setting is also generalised to the case without a unique dominated measure. Examples are provided where the new bound is compared with the classical Markov -- Dobrushin inequality and with the second eigenvalue of the transition matrix for finite state spaces.
In this paper, we give quantitative bounds on the $f$-total variation distance from convergence of an Harris recurrent Markov chain on an arbitrary under drift and minorisation conditions implying ergodicity at a sub-geometric rate. These bounds are then specialized to the stochastically monotone case, covering the case where there is no minimal reachable element. The results are illustrated on two examples from queueing theory and Markov Chain Monte Carlo.
We suggest an approach to obtaining general two-sided bounds on the rate of convergence in terms of special "weighted" norms related to total variation. Some important classes of continuous-time Markov chains are considered: birth-death-catastrophes processes, queueing models with batch arrivals and group services, chains with absorption in zero.
In the paper, we study a new rate of convergence estimate for homogeneous discrete-time nonlinear Markov chains based on the Markov-Dobrushin condition. This result generalizes the convergence estimates for any positive number of transition steps. An example of a class such a process provided indicates that such types of estimates considering several transition steps may be applicable when one transition can not guarantee any convergence. Moreover, a better estimate can be obtained for a higher number of transitions steps. A law of large numbers is presented for a class of ergodic nonlinear Ma…
We provide a abstract framework to prove total variation convergence result with arbitrary rate for numerical scheme for SDE. In particular we show that under standard weak approximation properties of scheme such as Euler we can obtain total variation convergence with any desired rate by building a specific approximation for the SDE.
In this note, a general approach to the study of non-stationary Markov chains with catastrophes and the corresponding queuing models is considered, as well as to obtain estimates of the limiting regime itself. As an illustration, an example of a queuing model is studied.
Our aim is to find sufficient conditions for weak convergence of stochastic integrals with respect to the state occupation measure of a Markov chain. First, we study properties of the state indicator function and the state occupation measure of a Markov chain. In particular, we establish weak convergence of the state occupation measure under a scaling of the generator matrix. Then, relying on the connection between the state occupation measure and the Dynkin martingale related to the state indicator function, we provide sufficient conditions for weak convergence of stochastic integrals with re…
This paper deals with three major types of convergence of probability measures on metric spaces: weak convergence, setwise converges, and convergence in the total variation. First, it describes and compares necessary and sufficient conditions for these types of convergence, some of which are well-known, in terms of convergence of probabilities of open and closed sets and, for the probabilities on the real line, in terms of convergence of distribution functions. Second, it provides % convenient criteria for weak and setwise convergence of probability measures and continuity of stochastic kernel…
We study the rate of weak convergence of Markov chains to diffusion processes under suitable but quite general assumptions. We give an example in the financial framework, applying the convergence analysis to a multiple jumps tree approximation of the CIR process. Then, we combine the Markov chain approach with other numerical techniques in order to handle the different components in jump-diffusion coupled models. We study the speed of convergence of this hybrid approach and we provide an example in finance, applying our results to a tree-finite difference approximation in the Heston or Bates m…
In this note, we are concerned with the subgeometric rate of convergence of a Markov chain with discrete time parameter to its invariant measure in the $f$-norm. We clarify how three typical subgeometric rates of convergence are inherited under a discrete time version of Bochner's subordination. The crucial point is to establish the corresponding moment estimates for discrete time subordinators under some reasonable conditions on the underlying Bernstein function.
Generalization of the Lorden's inequality is an excellent tool for obtaining strong upper bounds for the convergence rate for various complicated stochastic models. This paper demonstrates a method for obtaining such bounds for some generalization of the Markov modulated Poisson process (MMPP). The proposed method can be applied to reliability and queuing theory.
In this brief paper we find computable exponential convergence rates for a large class of stochastically ordered Markov processes. We extend the result of Lund, Meyn, and Tweedie (1996), who found exponential convergence rates for stochastically ordered Markov processes starting from a fixed initial state, by allowing for a random initial condition that is also stochastically ordered. Our bounds are formulated in terms of moment-generating functions of hitting times. To illustrate our result, we find an explicit exponential convergence rate for an M/M/1 queue beginning in equilibrium and then …
We provide explicit expressions for the constants involved in the characterisation of ergodicity of sub-geometric Markov chains. The constants are determined in terms of those appearing in the assumed drift and one-step minorisation conditions. The result is fundamental for the study of some algorithms where uniform bounds for these constants are needed for a family of Markov kernels. Our result accommodates also some classes of inhomogeneous chains.
We study the quasi-stationary behavior of multidimensional processes absorbed when one of the coordinates vanishes. Our results cover competitive or weakly cooperative Lotka-Volterra birth and death processes and Feller diffusions with competitive Lotka-Volterra interaction. To this aim, we develop original non-linear Lyapunov criteria involving two Lyapunov functions, which apply to general Markov processes.
In Latouche and Nguyen (2015), the authors constructed a sequence of stochastic fluid processes and showed that it converges weakly to a Markov-modulated Brownian motion (MMBM). Here, we construct a different sequence of stochastic fluid processes and show that it converges strongly to an MMBM. To the best of our knowledge, this is the first result on strong convergence to a Markov-modulated Brownian motion. We also prove that the rate of this almost sure convergence is $o(n^{-1/2} \log n)$. When reduced to the special case of standard Brownian motion, our convergence rate is an improvement ov…
We determine the convergence speed of a numerical scheme for approximating one-dimensional continuous strong Markov processes. The scheme is based on the construction of coin tossing Markov chains whose laws can be embedded into the process with a sequence of stopping times. Under a mild condition on the process' speed measure we prove that the approximating Markov chains converge at fixed times at the rate of $1/4$ with respect to every $p$-th Wasserstein distance. For the convergence of paths, we prove any rate strictly smaller than $1/4$. Our results apply, in particular, to processes with …
We provide necessary and sufficient conditions for convergence of exponential integrals of Markov additive processes. Other than in the classical Lévy case studied by Erickson and Maller we have to distinguish between almost sure convergence and convergence in probability. Our proofs rely on recent results on perpetuities in a Markovian environment by Alsmeyer and Buckmann.
We provide an original and general sufficient criterion ensuring the exponential contraction of Feynman-Kac semi-groups of penalized processes. This criterion is applied to time-inhomogeneous one-dimensional diffusion processes conditioned not to hit 0 and to penalized birth and death processes evolving in a quenched random environment.
The paper studies an improved estimate for the rate of convergence for nonlinear homogeneous discrete-time Markov chains. These processes are nonlinear in terms of the distribution law. Hence, the transition kernels are dependent on the current probability distributions of the process apart from being dependent on the current state. Such processes often act as limits for large-scale systems of dependent Markov chains with interaction. The paper generalizes the convergence results by taking the estimate over two steps. Such an approach keeps the existence and uniqueness results under assumption…
This paper considers an approximation usually used when implementing Ramaswami's recursion for the stationary distribution of the M/G/1-type Markov chain. The approximation is called the level-increment-truncation approximation because it truncates level increment at a given threshold. The main contribution of this paper is to present a geometric convergence formula of the level-wise difference between the respective stationary distributions of the original M/G/1-type Markov chain and its LI truncation approximation under the assumption that the level-increment distribution is light-tailed.
We analyse the structure of imprecise Markov chains and study their convergence by means of accessibility relations. We first identify the sets of states, so-called minimal permanent classes, that are the minimal sets capable of containing and preserving the whole probability mass of the chain. These classes generalise the essential classes known from the classical theory. We then define a class of extremal imprecise invariant distributions and show that they are uniquely determined by the values of the upper probability on minimal permanent classes. Moreover, we give conditions for unique con…
We are interested in the rate of convergence of a subordinate Markov process to its invariant measure. Given a subordinator and the corresponding Bernstein function (Laplace exponent) we characterize the convergence rate of the subordinate Markov process; the key ingredients are the rate of convergence of the original process and the (inverse of the) Bernstein function. At a technical level, the crucial point is to bound three types of moments (sub-exponential, algebraic and logarithmic) for subordinators as time $t$ tends to infinity. At the end we discuss some concrete models and we show tha…
Let $\{X_n\}_{n\in\N}$ be a Markov chain on a measurable space $\X$ with transition kernel $P$ and let $V:\X\r[1,+\infty)$. The Markov kernel $P$ is here considered as a linear bounded operator on the weighted-supremum space $\cB_V$ associated with $V$. Then the combination of quasi-compactness arguments with precise analysis of eigen-elements of $P$ allows us to estimate the geometric rate of convergence $ρ_V(P)$ of $\{X_n\}_{n\in\N}$ to its invariant probability measure in operator norm on $\cB_V$. A general procedure to compute $ρ_V(P)$ for discrete Markov random walks with identically dist…
We study the convergence rate to stationarity for a class of exchangeable partition-valued Markov chains called cut-and-paste chains. The law governing the transitions of a cut-and-paste chain are determined by products of i.i.d. stochastic matrices, which describe the chain induced on the simplex by taking asymptotic frequencies. Using this representation, we establish upper bounds for the mixing times of ergodic cut-and-paste chains, and under certain conditions on the distribution of the governing random matrices we show that the "cutoff phenomenon" holds.
Consider a filtering process associated to a hidden Markov model with densities for which both the state space and the observation space are complete, separable, metric spaces. If the underlying, hidden Markov chain is strongly ergodic and the filtering process fulfills a certain coupling condition we prove that, in the limit, the distribution of the filtering process is independent of the initial distribution of the hidden Markov chain. If furthermore the hidden Markov chain is uniformly ergodic, then we prove that the filtering process converges in distribution.
We consider a random walk with a negative drift and with a jump distribution which under Cramér's change of measure belongs to the domain of attraction of a spectrally positive stable law. If conditioned to reach a high level and suitably scaled, this random walk converges in law to a nondecreasing Markov process which can be interpreted as a spectrally-positive Lévy %-Khinchin process conditioned not to overshoot level one.
Studying the subexponential convergence towards equilibrium of a strong Markov process, we exhibit an intermediate Lyapunov condition equivalent to the control of some moment of a hitting time. This provides a link, similar (although more intricate) to the one existing in the exponential case, between the coupling method and the approach based on the existence of a Lyapunov function for the generator, in the context of the subexponential rates found by Fort-Roberts (2005), Douc-Fort-Guillin (2009) and Hairer (2016).
We consider a form of state-dependent drift condition for a general Markov chain, whereby the chain subsampled at some deterministic time satisfies a geometric Foster-Lyapunov condition. We present sufficient criteria for such a drift condition to exist, and use these to partially answer a question posed by Connor & Kendall (2007) concerning the existence of so-called 'tame' Markov chains. Furthermore, we show that our 'subsampled drift condition' implies the existence of finite moments for the return time to a small set.
Let $(X, Y) = (X_n, Y_n)_{n \geq 1}$ be the output process generated by a hidden chain $Z = (Z_n)_{n \geq 1}$, where $Z$ is a finite state, aperiodic, time homogeneous, and irreducible Markov chain. Let $LC_n$ be the length of the longest common subsequences of $X_1, \ldots, X_n$ and $Y_1, \ldots, Y_n$. Under a mixing hypothesis, a rate of convergence result is obtained for $\mathbb{E}[LC_n]/n$.
Convergence rates of Markov chains have been widely studied in recent years. In particular, quantitative bounds on convergence rates have been studied in various forms by Meyn and Tweedie [Ann. Appl. Probab. 4 (1994) 981-1101], Rosenthal [J. Amer. Statist. Assoc. 90 (1995) 558-566], Roberts and Tweedie [Stochastic Process. Appl. 80 (1999) 211-229], Jones and Hobert [Statist. Sci. 16 (2001) 312-334] and Fort [Ph.D. thesis (2001) Univ. Paris VI]. In this paper, we extend a result of Rosenthal [J. Amer. Statist. Assoc. 90 (1995) 558-566] that concerns quantitative convergence rates for time-homog…
Invariance principles are obtained for a Markov process on a half-line with continuous paths on the interior. The domains of attraction of the two different types of self-similar processes are investigated. Our approach is to establish convergence of excursion point processes, which is based on Itô's excursion theory and a recent result on convergence of excursion measures by Fitzsimmons and the present author.
We establish subgeometric bounds on convergence rate of general Markov processes in the Wasserstein metric. In the discrete time setting we prove that the Lyapunov drift condition and the existence of a "good" $d$-small set imply subgeometric convergence to the invariant measure. In the continuous time setting we obtain the same convergence rate provided that there exists a "good" $d$-small set and the Douc-Fort-Guillin supermartingale condition holds. As an application of our results, we prove that the Veretennikov-Khasminskii condition is sufficient for subexponential convergence of strong s…
In this paper it is shown that adaptive importance sampling algorithms converge at exponential rate for Markov chain expectation problems that admit a combination of a filtered estimator and a Markov zero-variance measure. It extends a chain of results---special purpose proofs were already known for several cases (Kollman et al, Baggerly et al, Desai). A recent paper (Awad et al) provides a complete description of the class of combinations of Markov process expectations of path functionals and filtered estimators that admit zero-variance importance measures that retain the Markov property. In …
Rate of convergence is studied for a diffusion process on the half line with a non-sticky reflection to a heavy-tailed 1D invariant distribution which density on the half line has a polynomial decay at infinity. Starting from a standard receipt which guarantees some polynomial convergence, it is shown how to construct a new non-degenerate diffusion process on the half line which converges to the same invariant measure exponentially fast uniformly with respect to the initial data.
Let $(S,ρ)$ be an ultrametric space with certain conditions and $S^k$ be the quotient space of $S$ with respect to the partition by balls with a fixed radius $ϕ(k)$. We prove that, for a Hunt process $X$ on $S$ associated with a Dirichlet form $(\mathcal E, \mathcal F)$, a Hunt process $X^k$ on $S^k$ associated with the averaged Dirichlet form $(\mathcal E^k, \mathcal F^k)$ is Mosco convergent to $X$, and under certain additional conditions, $X^k$ converges weakly to $X$. Moreover, we give a sufficient condition for the Markov property of $X$ to be preserved under the canonical projection $π^k…
We provide a sharp nonasymptotic analysis of the rates of convergence for some standard multivariate Markov chains using spectral techniques. All chains under consideration have multivariate orthogonal polynomial as eigenfunctions. Our examples include the Moran model in population genetics and its variants in community ecology, the Dirichlet-multinomial Gibbs sampler, a class of generalized Bernoulli--Laplace processes, a generalized Ehrenfest urn model and the multivariate normal autoregressive process.
In this work, we consider a finite-state inhomogeneous-time Markov chain whose probabilities of transition from one state to another tend to decrease over time. This can be seen as a cooling of the dynamics of an underlying Markov chain. We are interested in the long time behavior of the empirical measure of this freezing Markov chain. Some recent papers provide almost sure convergence and convergence in distribution in the case of the freezing speed $n^{-θ}$, with different limits depending on $θ<1,θ=1$ or $θ>1$. Using stochastic approximation techniques, we generalize these results for any f…
Let $(X_n)_{n=0}^\infty$ denote a Markov chain on a Polish space that has a stationary distribution $\varpi$. This article concerns upper bounds on the Wasserstein distance between the distribution of $X_n$ and $\varpi$. In particular, an explicit geometric bound on the distance to stationarity is derived using generalized drift and contraction conditions whose parameters vary across the state space. These new types of drift and contraction allow for sharper convergence bounds than the standard versions, whose parameters are constant. Application of the result is illustrated in the context of …
