The notion of a successful coupling of Markov processes, based on the idea that both components of the coupled system ``intersect'' in finite time with probability one, is extended to cover situations when the coupling is unnecessarily Markovian and its components are only converging (in a certain sense) to each other with time. Under these assumptions the unique ergodicity of the original Markov process is proven. A price for this generalization is the weak convergence to the unique invariant measure instead of the strong one. Applying these ideas to infinite interacting particle systems we consider even more involved situations when the unique ergodicity can be proven only for a restriction of the original system to a certain class of initial distributions (e.g. translational invariant ones). Questions about the existence of invariant measures with a given particle density are discussed as well.
Let $X$ be an irreducible symmetric Markov process with the strong Feller property. We assume, in addition, that $X$ is explosive and has a tightness property. We then prove the existence and uniqueness of quasi-stationary distributions of $X$.
We are interested in quasi-stationarity and quasi-ergodicity when the absorbing boundary is moving. First we show that, in the moving boundary case, the quasi-stationary distribution and the quasi-limiting distribution are not well-defined when the boundary is oscillating periodically. Then we show the existence of a quasi-ergodic distribution for any discrete-time irreducible Markov chain defined on a finite space state in the fixed boundary case. Finally we use this last result to show the quasi-ergodicity in the moving boundary case.
An algorithm for estimating quasi-stationary distribution of finite state space Markov chains has been proven in a previous paper. Now this paper proves a similar algorithm that works for general state space Markov chains under very general assumptions.
In this paper, we study quasi-stationary distributions of nonlinearly perturbed semi-Markov processes in discrete time. This type of distributions is of interest for the analysis of stochastic systems which have finite lifetimes, but are expected to persist for a long time. We obtain asymptotic power series expansions for quasi-stationary distributions and it is shown how the coefficients in these expansions can be computed from a recursive algorithm. As an illustration of this algorithm, we present a numerical example for a discrete time Markov chain.
We present formulas for quasi-ergodic limits of finite absorbing Markov chains. Since the irreducible case has been solved in 1965 by Darroch and Seneta, we focus on the reducible case, and our results are based on a very precise asymptotic analysis of the (exponential and polynomial) growth behaviour along admissible paths
We consider periodic Markov chains with absorption. Applying to iterates of this periodic Markov chain criteria for the exponential convergence of conditional distributions of aperiodic absorbed Markov chains, we obtain exponential estimates for the periodic asymptotic behavior of the semigroup of the Markov chain. This implies in particular the exponential convergence in total variation of the conditional distribution of the Markov chain given non-absorption to a periodic sequence of limit measures and we characterize the cases where this sequence is constant, which corresponds to the cases w…
For downward skip-free continuous-time Markov chains on non-negative integers stopped at zero, existence of a quasi-stationary distribution is studied. The scale function for these processes is introduced and the boundary is classified by a certain integrability condition on the scale function, which gives an extension of Feller's classification of the boundary for birth-and-death processes. The existence and the set of quasi-stationary distributions are characterized by the scale function and the new classification of the boundary.
In this paper, we give a sufficient condition for the existence of a quasi-ergodic distribution for absorbing Markov processes. Using an orthogonal-polynomial approach, we prove that the previous main result is valid for the birth-death process on the nonnegative integers with 0 an absorbing boundary and $\infty$ an entrance boundary. We also show that the quasi-ergodic distribution is stochastically larger than the unique quasi-stationary distribution in the sense of monotone likelihood-ratio ordering for the birth-death process.
For a fixed right process $X$ we investigate those functions $u$ for which $u(X)$ is a quasimartingale. We prove that $u(X)$ is a quasimartingale if and only if $u$ is the dif- ference of two finite excessive functions. In particular, we show that the quasimartingale nature of $u$ is preserved under killing, time change, or Bochner subordination. The study relies on an analytic reformulation of the quasimartingale property for $u(X)$ in terms of a certain variation of $u$ with respect to the transition function of the process. We provide sufficient conditions under which $u(X)$ is a quasimarti…
We identify the linear space spanned by the real-valued excessive functions of a Markov process with the set of those functions which are quasimartingales when we compose them with the process. Applications to semi-Dirichlet forms are given. We provide a unifying result which clarifies the relations between harmonic, co-harmonic, invariant, co-invariant, martingale and co-martingale functions, showing that in the conservative case they are all the same. Finally, using the co-excessive functions, we present a two-step approach to the existence of invariant probability measures.
Given a matrix of distribution functions and a quasi-stochastic matrix, i.e. an irreducible nonnegative matrix with maximal eigenvalue one and associated unique positive left and right eigenvectors, the article studies the properties of an associated matrix renewal measure and a related integral equation. Unlike earlier work this is done by a purely probabilistic approach based on a simple harmonic transform. Main results include Markov renewal-type theorems and a Stone-type decomposition under an absolute continuity condition. Three applications are given at the end of the paper.
For a relatively large class of well-behaved absorbing (or killed) finite Markov chains, we give detailed quantitative estimates regarding the behavior of the chain before it is absorbed (or killed). Typical examples are random walks on box-like finite subsets of the square lattice $\mathbb Z^d$ absorbed (or killed) at the boundary. The analysis is based on Poincaré, Nash, and Harnack inequalities, moderate growth, and on the notions of John and inner-uniform domains.
We consider a discrete time semi-Markov process where the characteristics defining the process depend on a small perturbation parameter. It is assumed that the state space consists of one finite communicating class of states and, in addition, one absorbing state. Our main object of interest is the asymptotic behaviour of the joint probabilities of the position of the semi-Markov process and the event of non-absorption as time tends to infinity and the perturbation parameter tends to zero. The main result gives exponential expansions of these probabilities together with an recursive algorithm f…
Let $P$ be a Markov kernel on a measurable space $\X$ and let $V:\X\r[1,+\infty)$. We provide various assumptions, based on drift conditions, under which $P$ is quasi-compact on the weighted-supremum Banach space $(\cB_V,\|\cdot\|_V)$ of all the measurable functions $f : \X\r\C$ such that $\|f\|_V := \sup_{x\in \X} |f(x)|/V(x) < \infty$. Furthermore we give bounds for the essential spectral radius of $P$. Under additional assumptions, these results allow us to derive the convergence rate of $P$ on $\cB_V$, that is the geometric rate of convergence of the iterates $P^n$ to the stationary distri…
We study the recurrence of inhomogeneous Markov chains in the plane, when the environment is horizontally stratified and the heterogeneity of quasi-periodic type.
We consider a Markov process $ X(t) $ on the nonnegative integers $E= S \cup \{0\}$, where $S=\{1,2,...\}$ is an irreducible class and 0 is an absorbing state. In this paper, we investigate conditions under which the quasi-stationary distribution for $X(t)$ exists and is unique, and any initial distribution supported in $S$ is in the domain of attraction of this quasi-stationary distribution. We further find five conditions which are equivalent to that the extinction time is uniformly bounded. As a consequence, we prove the van Doorn's conjecture in \cite{VD2012}. And we can greatly improve th…
We study the long time behaviour of a Markov process evolving in $\mathbb{N}$ and conditioned not to hit 0. Assuming that the process comes back quickly from infinity, we prove that the process admits a unique quasi-stationary distribution (in particular, the distribution of the conditioned process admits a limit when time goes to infinity). Moreover, we prove that the distribution of the process converges exponentially fast in total variation norm to its quasi-stationary distribution and we provide an explicit rate of convergence. As a first application of our result, we bring a new insight o…
We show how the essential spectral radius of a bounded positive kernel, acting on bounded functions, is linked to its lower approximation by certain absolutely continuous kernels. The standart Doeblin's condition can be interpreted in this context, and, when suitably reformulated, it leads to a formula for the essential spectral radius. This results may be used to characterize the Markov kernels having a quasi-compact action on a space of measurable functions bounded with respect to some test function, when no irreducibilty and aperiodicity are assumed.
We propose the study of Markov chains on groups as a "quasi-isometry invariant" theory that encompasses random walks. In particular, we focus on certain classes of groups acting on hyperbolic spaces including (non-elementary) hyperbolic and relatively hyperbolic groups, acylindrically hyperbolic 3-manifold groups, as well as fundamental groups of certain graphs of groups with edge groups of subexponential growth. For those, we prove a linear progress result and various applications, and these lead to a Central Limit Theorem for random walks on groups quasi-isometric to the ones we consider.
This paper deals with ergodic theorems for particular time-inhomogeneous Markov processes, whose the time-inhomogeneity is asymptotically periodic. Under a Lyapunov/minorization condition, it is shown that, for any measurable bounded function $f$, the time average $\frac{1}{t} \int_0^t f(X_s)ds$ converges in $\mathbb{L}^2$ towards a limiting distribution, starting from any initial distribution for the process $(X_t)_{t \geq 0}$. This convergence can be improved to an almost sure convergence under an additional assumption on the initial measure. This result will be then applied to show the exis…
Reinforced processes are known to provide a stochastic representation for the quasi-stationary distribution of a given killed Markov process - describing the killed Markov process at fixed time instants. In this paper we shall adapt the construction to provide a pathwise description. We also obtain a stochastic approximation for the quasi-limiting distributions of reducible killed Markov processes as a corollary.
This paper presents a new matrix-infinite-product-form (MIP-form) solution for the stationary distribution in upper block-Hessenberg Markov chains (UBH-MCs). The existing MIP-form solution (Masuyama, Queueing Syst., Vol. 92, 2019, pp. 173--200) requires a certain parameter set that satisfies both a Foster-Lyapunov drift condition and a convergence condition. In contrast, the new MIP-form solution requires no such parameter sets and no other conditions. The new MIP-form solution also has "quasi-algorithmic constructibility", which is a newly introduced feature of being constructed by iterating …
Let $(X,\p_x)$ be a continuous time Markov chain with finite or countable state space $S$ and let $T$ be its first passage time in a subset $D$ of $S$. It is well known that if $μ$ is a quasi-stationary distribution relatively to $T$, then this time is exponentially distributed under $\p_μ$. However, quasi-stationarity is not a necessary condition. In this paper, we determine more general conditions on an initial distribution $μ$ for $T$ to be exponentially distributed under $\p_μ$. We show in addition how quasi-stationary distributions can be expressed in terms of any initial law which makes …
Let {M_n}_{n\ge 0}$ be a nonnegative Markov process with stationary transition probabilities. The quasistationary distributions referred to in this note are of the form Q_A(x) = lim_{n\to\infty} P(M_n \le x | M_0 \le A, M_1 \le A, ..., M_n \le A) . Suppose that $M_0$ has distribution $\Qb_A$ and define T_A^{Q_A} = \min\{n | M_n > A, n\ge 1\}, the first time when M_n exceeds A. We provide sufficient conditions for E T_A^{Q_A}$ to be an increasing function of A.
We are interested in the connection between a metastable continuous state space Markov process (satisfying e.g. the Langevin or overdamped Langevin equation) and a jump Markov process in a discrete state space. More precisely, we use the notion of quasi-stationary distribution within a metastable state for the continuous state space Markov process to parametrize the exit event from the state. This approach is useful to analyze and justify methods which use the jump Markov process underlying a metastable dynamics as a support to efficiently sample the state-to-state dynamics (accelerated dynami…
In this paper we develop the theory of {\it polymorphisms} of measure spaces, which is a generalization of the theory of measure-preserving transformations; we describe the main notions and discuss relations to the theory of Markov processes, operator theory, ergodic theory, etc. Using these tools, we solve the problem, which appeared in dynamics in the 70s: what kind of equivalence can exist between deterministic and random dynamical systems; our refining of this question is as follows: is it possible to have the quasi-similarity of a measure-preserving automorphism and a polymorphism. We pro…
A Markov operator $P$ on a probability space $(S,Σ,μ)$, with $μ$ invariant, is called {\it hyperbounded} if for some $1 \le p<q \le \infty$ it maps (continuously) $L^p$ into $L^q$. We deduce from a recent result of Glück that a hyperbounded $P$ is quasi-compact, hence uniformly ergodic, in all $L^r(S,μ)$, $1<r< \infty$. We prove, using a method similar to Foguel's, that a hyperbounded Markov operator has periodic behavior similar to that of Harris recurrent operators, and for the ergodic case obtain conditions for aperiodicity. Given a probability $ν$ on the unit circle, we prove that if the c…
We study Markov population processes on large graphs, with the local state transition rates of a single vertex being linear function of its neighborhood. A simple way to approximate such processes is by a system of ODEs called the homogeneous mean-field approximation (HMFA). Our main result is showing that HMFA is guaranteed to be the large graph limit of the stochastic dynamics on a finite time horizon if and only if the graph-sequence is quasi-random. Explicit error bound is given and being of order $\frac{1}{\sqrt{N}}$ plus the largest discrepancy of the graph. For Erdős Rényi and random re…
We investigate some asymptotic properties of general Markov processes conditioned not to be absorbed by moving boundaries. We first give general criteria involving an exponential convergence towards the Q-process, that is the law of the considered Markov process conditioned never to reach the moving boundaries. This exponential convergence allows us to state the existence and uniqueness of quasi-ergodic distribution considering either boundaries moving periodically or stabilizing boundaries. We also state the existence and uniqueness of quasi-limit distribution when absorbing boundaries stabil…
In the continuity of a recent paper ([6]), dealing with finite Markov chains, this paper proposes and analyzes a recursive algorithm for the approximation of the quasi-stationary distribution of a general Markov chain living on a compact metric space killed in finite time. The idea is to run the process until extinction and then to bring it back to life at a position randomly chosen according to the (possibly weighted) empirical occupation measure of its past positions. General conditions are given ensuring the convergence of this measure to the quasi-stationary distribution of the chain. We t…
Quasi-stationary distributions (QSD) have been widely studied since the pioneering work of Kolmogorov (1938), Yaglom (1947) and Sevastyanov (1951). They appear as a natural object when considering Markov processes that are certainly absorbed since they are invariant for the evolution of the distribution of the process conditioned on not being absorbed. They hence appropriately describe the state of the process at large times for non absorbed paths. Unlike invariant distributions for Markov processes, QSD are solutions of a non-linear equation and there can be 0, 1 or an infinity of them. Also,…
Quasi-Monte Carlo (QMC) methods for estimating integrals are attractive since the resulting estimators typically converge at a faster rate than pseudo-random Monte Carlo. However, they can be difficult to set up on arbitrary posterior densities within the Bayesian framework, in particular for inverse problems. We introduce a general parallel Markov chain Monte Carlo (MCMC) framework, for which we prove a law of large numbers and a central limit theorem. In that context, non-reversible transitions are investigated. We then extend this approach to the use of adaptive kernels and state conditions…
Let $Q$ be a transition probability on a measurable space $E$, let $(X\_n)\_n$ be a Markov chain associated to $Q$, and let $ξ$ be a real-valued measurable function on $E$, and $S\_n = \sum\_{k=1}^{n} ξ(X\_k)$. Under functional hypotheses on the action of $Q$ and its Fourier kernels $Q(t)$, we investigate the rate of convergence in the central limit theorem for the sequence $(\frac{S\_n}{\sqrt n})\_n$. According to the hypotheses, we prove that the rate is, either $O(n^{-\fracτ{2}})$ for all $τ<1$, or $O(n^{-{1/2}})$. We apply the spectral method of Nagaev which is improved by using a perturba…
We study infinite horizon control of continuous-time non-linear branching processes with almost sure extinction for general (positive or negative) discount. Our main goal is to study the link between infinite horizon control of these processes and an optimization problem involving their quasi-stationary distributions and the corresponding extinction rates. More precisely, we obtain an equivalent of the value function when the discount parameter is close to the threshold where the value function becomes infinite, and we characterize the optimal Markov control in this limit. To achieve this, we …
The quantitative long time behavior of absorbing, finite, irreducible Markov processes is considered. Via Doob transforms, it is shown that only the knowledge of the ratio of the values of the underlying first Dirichlet eigenvector is necessary to come back to the well-investigated situation of the convergence to equilibrium of ergodic finite Markov processes. This leads to explicit estimates on the convergence to quasi-stationarity, in particular via functional inequalities. When the process is reversible, the optimal exponential rate consisting of the spectral gap between the two first Diric…
This paper considers space homogenous Boltzmann kinetic equations in dimension $d$ with Maxwell collisions (and without Grad's cut-off). An explicit Markov coupling of the associated conservative (Nanbu) stochastic $N$-particle system is constructed, using plain parallel coupling of isotropic random walks on the sphere of two-body collisional directions. The resulting coupling is almost surely decreasing, and the $L_2$-coupling creation is computed explicitly. Some quasi-contractive and uniform in $N$ coupling / coupling creation inequalities are then proved, relying on $2+α$-moments ($α>0$) o…
This paper aims to provide some tools coming from functional inequalities to deal with quasi-stationarity for absorbed Markov processes. First, it is shown how a Poincaré inequality related to a suitable Doob transform entails exponential convergence of conditioned distributions to a quasi-stationary distribution in total variation and in $1$-Wasserstein distance. A special attention is paid to multi-dimensional diffusion processes, for which the aforementioned Poincaré inequality is implied by an easier-to-check Bakry-Émery condition depending on the right eigenvector for the sub-Markovian ge…
For a continuous-time Markov process, we characterize the law of the first jump location when started from an arbitrary initial distribution, in terms of the invariant distribution of an auxiliary Markov process. This could be of interest in the burgeoning fields of piecewise-deterministic Markov chain Monte Carlo methods and quasi-stationary Monte Carlo methods.
This paper aims to establish a central limit theorem for Markov processes conditioned not to be absorbed under a very general assumption on quasi-stationarity for the underlying process. To do so, a central limit theorem has been established for ergodic Markov processes. The conditional central limit theorem is then obtained by applying the central limit theorem to the $Q$-process.
We study the quasi-stationary behavior of multidimensional processes absorbed when one of the coordinates vanishes. Our results cover competitive or weakly cooperative Lotka-Volterra birth and death processes and Feller diffusions with competitive Lotka-Volterra interaction. To this aim, we develop original non-linear Lyapunov criteria involving two Lyapunov functions, which apply to general Markov processes.
Motivated by tumor growth models we establish conditions for the $R-$positivity of Markov processes and positive matrices. We then apply them to obtain the asymptotic behavior of the tumors sizes in the supercritical regime.
We discuss the existence and characterization of quasi-stationary distributions and Yaglom limits of self-similar Markov processes that reach 0 in finite time. By Yaglom limit, we mean the existence of a deterministic function $g$ and a non-trivial probability measure $ν$ such that the process rescaled by $g$ and conditioned on non-extinction converges in distribution towards $ν$. If the study of quasi-stationary distributions is easy and follows mainly from a previous result by Bertoin and Yor \cite{BYFacExp} and Berg \cite{bergI}, that of Yaglom limits is more challenging. We will see that a…
In this note we study the 2d stochastic quasi-geostrophic equation in $\mathbb{T}^2$ for general parameter $α\in (0,1)$ and multiplicative noise. We prove the existence of martingale solutions and pathwise uniqueness under some condition in the general case, i.e. for all $α\in (0,1)$. In the subcritical case $α>1/2$, we prove existence and uniqueness of (probabilistically) strong solutions and construct a Markov family of solutions. In particular, it is uniquely ergodic for $α>2/3$ provided the noise is non-degenerate. In this case, the convergence to the (unique) invariant measure is exponent…
Consider a one-sided Markov additive process with an upper and a lower barrier, where each can be either reflecting or terminating. For both defective and non-defective processes and all possible scenarios we identify the corresponding potential measures, which generalizes a number of results for one-sided Levy processes. The resulting rather neat formulas have various applications, and in particular they lead to quasi-stationary distributions of the corresponding processes.
