Finite Markov chains coupled to general Markov processes and an application to metastability I
Thomas G. Kurtz, Jason Swanson
备注：
15 pages; to be published in "Stochastic Analysis, Filtering, and Stochastic Optimization: A Commemorative Volume to Honor Mark H. A. Davis's Contributions"
We consider a diffusion given by a small noise perturbation of a dynamical system driven by a potential function with a finite number of local minima. The classical results of Freidlin and Wentzell show that the time this diffusion spends in the domain of attraction of one of these local minima is approximately exponentially distributed and hence the diffusion should behave approximately like a Markov chain on the local minima. By the work of Bovier and collaborators, the local minima can be associated with the small eigenvalues of the diffusion generator. Applying a Markov mapping theorem, we use the eigenfunctions of the generator to couple this diffusion to a Markov chain whose generator has eigenvalues equal to the eigenvalues of the diffusion generator that are associated with the local minima and establish explicit formulas for conditional probabilities associated with this coupling. The fundamental question then becomes to relate the coupled Markov chain to the approximate Markov chain suggested by the results of Freidlin and Wentzel. In Part II of this work, we provide a complete analysis of this relationship in the special case of a double-well potential in one dimension. More generally, the coupling can be constructed for a general class of Markov processes and any finite set of eigenvalues of the generator.
We propose a new definition of metastability of Markov processes on countable state spaces. We obtain sufficient conditions for a sequence of processes to be metastable. In the reversible case these conditions are expressed in terms of the capacity and of the stationary measure of the metastable states.
Consider a sequence of continuous-time Markov chains $(X^{(n)}_t:t\ge 0)$ evolving on a fixed finite state space $V$. Let $I_n$ be the level two large deviations rate functional for $X^{(n)}_t$, as $t\to\infty$. Under a hypothesis on the jump rates, we prove that $I_n$ can be written as $I_n = I^{(0)} \,+\, \sum_{1\le p\le q} (1/θ^{(p)}_n) \, I^{(p)}$ for some rate functionals $I^{(p)}$. The weights $θ^{(p)}_n$ correspond to the time-scales at which the sequence of Markov chains $X^{(n)}_t$ exhibit a metastable behavior, and the zero level sets of the rate functionals $I^{(p)}$ identify the me…
We prove the metastable behavior of reversible Markov processes on finite state spaces under minimal conditions on the jump rates. To illustrate the result we deduce the metastable behavior of the Ising model with a small magnetic field at very low temperature.
We proposed in \cite{bl2} a new approach to prove the metastable behavior of reversible dynamics based on potential theory and local ergodicity. In this article we extend this theory to nonreversible dynamics based on the Dirichlet principle proved in \cite{gl2}.
We consider a diffusion given by a small noise perturbation of a dynamical system driven by a potential function with a finite number of local minima. The classical results of Freidlin and Wentzell show that the time this diffusion spends in the domain of attraction of one of these local minima is approximately exponentially distributed and hence the diffusion should behave approximately like a Markov chain on the local minima. By the work of Bovier and collaborators, the local minima can be associated with the small eigenvalues of the diffusion generator. In Part I of this work, by applying a…
Consider a sequence $(η^N(t) :t\ge 0)$ of continuous-time, irreducible Markov chains evolving on a fixed finite set $E$, indexed by a parameter $N$. Denote by $R_N(η,ξ)$ the jump rates of the Markov chain $η^N_t$, and assume that for any pair of bonds $(η,ξ)$, $(η',ξ')$ $\arctan \{R_N(η,ξ)/R_N(η',ξ')\}$ converges as $N\uparrow\infty$. Under a hypothesis slightly more restrictive (cf. \eqref{mhyp} below), we present a recursive procedure which provides a sequence of increasing time-scales $θ^1_N, \dots, θ^{\mf p}_N$, $θ^j_N \ll θ^{j+1}_N$, and of coarsening partitions $\{\ms E^j_1, \dots, \ms E…
Metastability is a physical phenomenon ubiquitous in first order phase transitions. A fruitful mathematical way to approach this phenomenon is the study of rare transitions Markov chains. For Metropolis chains associated with Statistical Mechanics systems, this phenomenon has been described in an elegant way in terms of the energy landscape associated to the Hamiltonian of the system. In this paper, we provide a similar description in the general rare transitions setup. Beside their theoretical content, we believe that our results are a useful tool to approach metastability for non--Metropolis…
Large continuous-time Markov chains with exponentially small transition rates arise in modeling complex systems in physics, chemistry and biology. We propose a constructive graph-algorithmic approach to determine the sequence of critical timescales at which the qualitative behavior of a given Markov chain changes, and give an effective description of the dynamics on each of them. This approach is valid for both time-reversible and time-irreversible Markov processes, with or without symmetry. Central to this approach are two graph algorithms, Algorithm 1 and Algorithm 2, for obtaining the seque…
We consider a random walk with catastrophes which was introduced to model population biology. It is known that this Markov chain gets eventually absorbed at $0$ for all parameter values. Recently, it has been shown that this chain exhibits a metastable behavior in the sense that it can persist for a very long time before getting absorbed. In this paper we study this metastable phase by making the parameters converge to extreme values. We obtain four different limits that we believe shed light on the metastable phase.
A family $\{Q_β\}_{β\geq 0}$ of Markov chains is said to exhibit $\textit{metastable mixing}$ with $\textit{modes}$ $S_β^{(1)},\ldots,S_β^{(k)}$ if its spectral gap (or some other mixing property) is very close to the worst conductance $\min(Φ_β(S_β^{(1)}), \ldots, Φ_β(S_β^{(k)}))$ of its modes. We give simple sufficient conditions for a family of Markov chains to exhibit metastability in this sense, and verify that these conditions hold for a prototypical Metropolis-Hastings chain targeting a mixture distribution. Our work differs from existing work on metastability in that, for the class of …
We presented in \cite{bl2,bl7} an approach to derive the metastable behavior of continuous-time Markov chains. We assumed in these articles that the Markov chains visit points in the time scale in which it jumps among the metastable sets. We replace this condition here by assumtpions on the mixing times and on the relaxation times of the chains reflected at the boundary of the metastable sets.
We study a large class of reversible Markov chains with discrete state space and transition matrix $P_N$. We define the notion of a set of {\it metastable points} as a subset of the state space $\G_N$ such that (i) this set is reached from any point $x\in \G_N$ without return to x with probability at least $b_N$, while (ii) for any two point x,y in the metastable set, the probability $T^{-1}_{x,y}$ to reach y from x without return to x is smaller than $a_N^{-1}\ll b_N$. Under some additional non-degeneracy assumption, we show that in such a situation: \item{(i)} To each metastable point corres…
Non-trivial topological behavior appears in many different contexts in statistical physics, perhaps the most known one being the Kosterlitz-Thouless phase transition in the two dimensional XY model. We study the behavior of a simpler, one dimensional, XY chain with periodic boundary and strong interactions; but rather than concentrating on the equilibrium measure we try to understand its dynamics. The equivalent of the Kosterlitz-Thouless transition in this one dimensional case happens when the interaction strength scales like the size of the system $N$, yet we show that a sharp transition for…
We study the performance of Markov chains for the $q$-state ferromagnetic Potts model on random regular graphs. It is conjectured that their performance is dictated by metastability phenomena, i.e., the presence of "phases" (clusters) in the sample space where Markov chains with local update rules, such as the Glauber dynamics, are bound to take exponential time to escape. The phases that are believed to drive these metastability phenomena in the case of the Potts model emerge as local, rather than global, maxima of the so-called Bethe functional, and previous approaches of analysing these pha…
We consider small perturbations of a dynamical system on the one-dimensional torus. We derive sharp estimates for the pre-factor of the stationary state, we examine the asymptotic behavior of the solutions of the Hamilton-Jacobi equation for the pre-factor, we compute the capacities between disjoint sets, and we prove the metastable behavior of the process among the deepest wells following the martingale approach. We also present a bound for the probability that a Markov process hits a set before some fixed time in terms of the capacity of an enlarged process.
These notes constitute the introduction to potential theory I exposed at the XIIth brazilian school of probability inside Elisabetta Scoppola's Introduction to Metastability.
We consider a general class of finite dimensional deterministic dynamical systems with finitely many local attractors $K^i$ each of which supports a unique ergodic probability measure $P^i$, which includes in particular the class of Morse-Smale systems in any finite dimension. The dynamical system is perturbed by a multiplicative non-Gaussian heavy-tailed Lévy type noise of small amplitude $\varepsilon>0$. Specifically we consider perturbations leading to a Itô, Stratonovich and canonical (Marcus) stochastic differential equation. The respective asymptotic first exit time and location problem …
This article continues our study of Markovian consistency and Markov copulae. In particular, we characterize the weak Markovian consistency for finite Markov chains. We discuss some aspects of dependence between the components of a multivariate Markov chain in the context of weak Markovian consistency and strong Markovian consistency. In this connection, we also introduce and discuss the concept of weak Markov copulae.
We consider the Markov chain approximations for singular stable-like processes. First we obtain properties of some Markov chains. Then we construct the approximating Markov chains and give a necessary condition for weak convergence of these chains to singular stable-like processes.
Given a strongly stationary Markov chain and a finite set of stopping rules, we prove the existence of a polynomial algorithm which projects the Markov chain onto a minimal Markov chain without redundant information. Markov complexity is hence defined and tested on some classical problems.
We consider a class of continuous time Markov chains on $\Z^d$. These chains are the discrete space analogue of Markov processes with jumps. Under some conditions, we show that harmonic functions associated with these Markov chains are Hölder continuous.
We consider the higher-order Markov Chain, and characterize the second order Markov chains admitting every probability distribution vector as a stationary vector. The result is used to construct Markov chains of higher-order with the same property. We also study conditions under which the set of stationary vectors of the Markov chain has a certain affine dimension.
All one-condition generalized inverses of the Markovian kernel I - P, where P is the transition matrix of a finite irreducible Markov chain, can be uniquely specified in terms of the stationary probabilities and the mean first passage times of the underlying Markov chain. Special sub-families include the group inverse of I - P, Kemeny and Snell's fundamental matrix of the Markov chain and the Moore-Penrose g-inverse. The elements of some sub-families of the generalized inverses can also be re-expressed involving the second moments of the recurrence time variables. Some applications to Kemeny's…
The asymptotic variance is an important criterion to evaluate the performance of Markov chains, especially for the central limit theorems. We give the variational formulas for the asymptotic variance of discrete-time (non-reversible) Markov chains on general state space. The variational formulas provide many applications, extending the classical Peskun's comparison theorem to non-reversible Markov chains, and obtaining several comparison theorems between Markov chains with various perturbations.
In this paper we discuss weak convergence of continuous-time Markov chains to a non-symmetric pure jump process. We approach this problem using Dirichlet forms as well as semimartingales. As an application, we discuss how to approximate a given Markov process by Markov chains.
In this paper we continue the study of conditional Markov chains (CMCs) with finite state spaces, that we initiated in Bielecki, Jakubowski and Niewęgłowski (2015). Here, we turn our attention to the study of Markov consistency and Markov copulae with regard to CMCs, and thus we follow up on the study of Markov consistency and Markov copulae for ordinary Markov chains that we presented in Bielecki, Jakubowski and Niewęgłowski (2013).
We study inhomogeneous continuous-time weakly ergodic Markov chains with a finite state space. We introduce the notion of a Markov chain with the regular structure of an infinitesimal matrix and study the sharp upper bounds on the rate of convergence for such class of Markov chains.
In this paper, we present reversibility preserving operations on Markov chain transition matrices. Simple row and column operations allow us to create new reversible transition matrices and yield an easy method for checking a Markov chain for reversibility.
We define fully coupled forward-backward stochastic differential equations on spaces related to continuous time, finite state Markov Chains. Existence and uniqueness results of the fully coupled forward-backward stochastic differential equations on Markov Chains are obtained.
The main purpose of this work is to study self-similar branching Markov chains. First we will construct such a process. Then we will establish certain Limit Theorems using the theory of self-similar Markov processes.
We prove that an irreducible aperiodic Markov chain is geometrically ergodic if and only if any separately bounded functional of the stationary chain satisfies an appropriate subgaussian deviation inequality from its mean.
This paper studies the augmented truncation of discrete-time block-monotone Markov chains under geometric drift conditions. We first present a bound for the total variation distance between the stationary distributions of an original Markov chain and its augmented truncation. We also obtain such error bounds for more general cases where an original Markov chain itself may not be block-monotone but is block-wise dominated by a block-monotone Markov chain. Finally we discuss the application of our results to GI/G/1-type Markov chains.
Extensions of Kemeny's constant, as derived for irreducible finite Markov chains in discrete time, to Markov renewal processes and Markov chains in continuous time are discussed. Three alternative Kemeny's functions and their variants are considered. Typically, they lead to a constant if and only if the mean holding times between the states in the Markov renewal process are constant. However one particular variant leads to a constant, analogous to the discrete time Markov chain result. Specifically, if the state space is finite, the weighted sum of the mean first passage times (omitting the me…
We consider the problem of characterising expected hitting times and hitting probabilities for imprecise Markov chains. To this end, we consider three distinct ways in which imprecise Markov chains have been defined in the literature: as sets of homogeneous Markov chains, as sets of more general stochastic processes, and as game-theoretic probability models. Our first contribution is that all these different types of imprecise Markov chains have the same lower and upper expected hitting times, and similarly the hitting probabilities are the same for these three types. Moreover, we provide a ch…
We consider periodic Markov chains with absorption. Applying to iterates of this periodic Markov chain criteria for the exponential convergence of conditional distributions of aperiodic absorbed Markov chains, we obtain exponential estimates for the periodic asymptotic behavior of the semigroup of the Markov chain. This implies in particular the exponential convergence in total variation of the conditional distribution of the Markov chain given non-absorption to a periodic sequence of limit measures and we characterize the cases where this sequence is constant, which corresponds to the cases w…
We study an approximation method of stationary characters of a two-dimensional Markov chain via the Stein method. For this purpose, innovative methods are developed to estimate the moments of the Markov chain, as well as the solution to the Poisson equation with a partial differential operator.
Let $(ξ_n)_{n=0}^\infty$ be a nonhomogeneous Markov chain taking values from finite state-space of $\mathbf{X}=\{1,2,\ldots,b\}$. In this paper, we will study the generalized entropy ergodic theorem with almost-everywhere and $\mathcal{L}_1$ convergence for nonhomogeneous Markov chains, which generalizes the corresponding classical results for the Markov chains.
In this paper, we quantify some known approximation to the Curie-Weiss model via applying the Stein method to the Markov chain whose stationary distribution coincides with Curie-Weiss model.
We are interested in studying the sensitivity of diffusion processes or their approximations by Markov Chains with respect to a perturbation of the coefficients.
In many dynamical systems in nature, the law of the dynamics changes along with the temporal evolution of the system. These changes are often associated with the occurrence of certain events. The timing of occurrence of these events depends, in turn, on the trajectory of the dynamical system itself, making the dynamics of the system and the timing of a change in the dynamics strongly coupled. Naturally, trajectories that take longer to satisfy the event will last longer. Therefore, we expect to observe more frequently the dominant dynamics, the ones that take longer to change in the long run. …
Starting from a Markov chain with a finite alphabet, we consider the chain obtained when all but one symbol are undistinguishable for the practitioner. We study necessary and sufficient conditions for this chain to have continuous transition probabilities with respect to the past.
Improved rates of convergence for ergodic Markov chains and relaxed conditions for them, as well as analogous convergence results for some non-homogeneous Markov chains are studied. The setting from the previous works is extended. Examples are provided where the new bounds are better and where they give the same convergence rate as in the classical Markov -- Dobrushin inequality.
In this paper we study the almost sure central limit theorem started from a point for additive functionals of a stationary and ergodic Markov chain via a martingale approximation in the almost sure sense. As a consequence we derive the quenched CLT for additive functionals or reversible Markov chains.
Stemmed from the derivation of the optimal control to a stochastic linear-quadratic control problem with Markov jumps, we study one kind of backward stochastic differential equations (BSDEs) that the generator f is affected by a Markovian switching. Then, the case that the Markov chain is involved in a large state space is considered. Following the classical approach, a hierarchical approach is adopted to reduce the complexity and a singularly perturbed Markov chain is involved. We will study the asymptotic property of BSDE with the singularly perturbed Markov chain. At last, as an application…
This is the story of the encounter between two worlds: the world of random walks and the world of Variable Length Markov Chains (VLMC). The meeting point turns around the semi-Markov property of underlying processes.
In this paper we are concerned with hitting times of a family of density-dependent Markov chains. A moderate deviation principle of the hitting time is given. The proof of the main theorem relies heavily on moderate deviations of density-dependent Markov chains given in \cite{Xue2021} and upper bounds of large deviations of Markov processes given in \cite{Dupuis1991}. An analogue moderate deviation of the hitting time of the diffusion approximation of the density-dependent Markov chain introduced in \cite{Ethier1986} is also given.
We consider a time-homogeneous Markov chain $X_n$, $n\ge0$, valued in ${\bf R}$. Suppose that this chain is transient, that is, $X_n$ generates a $σ$-finite renewal measure. We prove the key renewal theorem under condition that this chain has asymptotically homogeneous at infinity jumps and asymptotically positive drift.
We study certain properties of the function space of autocorrelation functions of Unit Continuous Time Markov Chains (CTMCs). It is shown that under particular conditions, the $L^p$ norm of the autocorrelation function of arbitrary finite state space CTMCs is infinite. Several interesting inferences are made for point processes associated with CTMCs/ Discrete Time Markov Chains (DTMCs).
In this paper we study the additive functionals of Markov chains via conditioning with respect to both past and future of the chain. We shall point out new sufficient projective conditions, which assure that the variance of partial sums of n consecutive random variables of a stationary Markov chain is linear in n. The paper also addresses the central limit theorem problem and is listing several open questions.
On conditions for Dobrushin's Central limit theorem for non-homogeneous Markov chains
A.A. Kharkevich Institute for Information Transmission Problems of Russian Academy of Sciences（俄罗斯科学院信息传输问题阿·阿·哈尔维奇研究所）；S.M. Nikol’skii Mathematical Institute, RUDN University（俄罗斯人民友谊大学尼古利斯基数学研究所）；National Research University Higher School of Economics（国立高等经济大学）
The present paper establishes a discrete version of the result obtained by P. Carr and S. Nadtochiy (2011) for 1-dimensional diffusion processes. Our result is for Markov chains on $\mathbf{Z}^d$.
