We take on a Random Matrix theory viewpoint to study the spectrum of certain reversible Markov chains in random environment. As the number of states tends to infinity, we consider the global behavior of the spectrum, and the local behavior at the edge, including the so called spectral gap. Results are obtained for two simple models with distinct limiting features. The first model is built on the complete graph while the second is a birth-and-death dynamics. Both models give rise to random matrices with non independent entries.
We propose some backward-forward martingale decompositions for functions of reversible Markov chains. These decompositions are used to prove the functional CLT for reversible Markov chains with asymptotically linear variance of partial sums. We also provide a proof of the equivalence between asymptotic linearity of the variance and convergence of the integral of $1/(1-t)$ with respect to the associated spectral measure $ρ$. We also study the asymptotic behavior of linear processes having as innovations mean zero square integrable functions of stationary reversible Markov chains. We apply this …
It has been well known for some time that for strictly stationary Markov chains that are ``reversible'', that special symmetry provides special extra features in the mathematical theory. This paper here is primarily a purely expository review of some of the basic features of that special theory. The mathematical techniques employed in this review are relatively gentle, involving only some basic measure-theoretic probability theory.
In this paper, we develop an in-depth analysis of non-reversible Markov chains on denumerable state space from a similarity orbit perspective. In particular, we study the class of Markov chains whose transition kernel is in the similarity orbit of a normal transition kernel, such as the one of birth-death chains or reversible Markov chains. We start by identifying a set of sufficient conditions for a Markov chain to belong to the similarity orbit of a birth-death one. As by-products, we obtain a spectral representation in terms of non-self-adjoint resolutions of identity in the sense of Dunfor…
A class of examples is constructed to show that for strictly stationary Markov chains that are reversible, the simultaneous mixing rates for the $ρ$-mixing and strong mixing ($α$-mixing) conditions can be fairly arbitrary, within certain unavoidable tight restrictions. The examples constructed here have the added property that the mixing rate for the absolute regularity ($β$-mixing) condition is within a constant factor of that for strong mixing.
In this paper, we investigate the functional central limit theorem for stochastic processes associated to partial sums of additive functionals of reversible Markov chains with general spate space, under the normalization standard deviation of partial sums. For this case, we show that the functional central limit theorem is equivalent to the fact that the variance of partial sums is regularly varying with exponent 1 and the partial sums satisfy the CLT. It is also equivalent to the conditional CLT.
A variational formula for the asymptotic variance of general Markov processes is obtained. As application, we get a upper bound of the mean exit time of reversible Markov processes, and some comparison theorems between the reversible and non-reversible diffusion processes.
We present a Markov chain example where non-reversibility and an added edge jointly improve mixing time: when a random edge is added to a cycle of $n$ vertices and a Markov chain with a drift is introduced, we get mixing time of $O(n^{3/2})$ with probability bounded away from 0. If only one of the two modifications were performed, the mixing time would stay $Ω(n^2)$.
We prove a complete class theorem that characterizes \emph{all} stationary time reversible Markov processes whose finite dimensional marginal distributions (of all orders) are infinitely divisible. Aside from two degenerate cases (iid and constant), in both discrete and continuous time every such process with full support is a branching process with Poisson or Negative Binomial marginal univariate distributions and a specific bivariate distribution at pairs of times. As a corollary, we prove that every nondegenerate stationary integer valued processes constructed by the Markov thinning process…
Markov chain Monte Carlo(MCMC) is a popular approach to sample from high dimensional distributions, and the asymptotic variance is a commonly used criterion to evaluate the performance. While most popular MCMC algorithms are reversible, there is a growing literature on the development and analyses of nonreversible MCMC. Chen and Hwang(2013) showed that a reversible MCMC can be improved by adding an antisymmetric perturbation. They also raised a conjecture that it can not be improved if there is no cycle in the corresponding graph. In this paper, we present a rigorous proof of this conjecture. …
It is shown that transient graphs for the simple random walk do not admit a nearest neighbor transient Markov chain (not necessarily a reversible one), that crosses all edges with positive probability, while there is such chain for the square grid $\mathbb{Z}^2$. In particular, the $d$-dimensional grid $\mathbb{Z}^d$ admits such a Markov chain only when $d=2$. For $d=2$ we present a relevant example due to Gady Kozma, while the general statement for transient graphs is obtained by proving that for every transient irreducible Markov chain on a countable state space, which admits a stationary me…
Consider a sequence of continuous-time Markov chains $(X^{(n)}_t:t\ge 0)$ evolving on a fixed finite state space $V$. Let $I_n$ be the level two large deviations rate functional for $X^{(n)}_t$, as $t\to\infty$. Under a hypothesis on the jump rates, we prove that $I_n$ can be written as $I_n = I^{(0)} \,+\, \sum_{1\le p\le q} (1/θ^{(p)}_n) \, I^{(p)}$ for some rate functionals $I^{(p)}$. The weights $θ^{(p)}_n$ correspond to the time-scales at which the sequence of Markov chains $X^{(n)}_t$ exhibit a metastable behavior, and the zero level sets of the rate functionals $I^{(p)}$ identify the me…
We present results relating mixing times to the intersection time of branching random walk (BRW) in which the logarithm of the expected number of particles grows at rate of the spectral-gap $\mathrm{gap}$ . This is a finite state space analog of a critical branching process. Namely, we show that the maximal expected hitting time of a state by such a BRW is up to a universal constant larger than the $L_{\infty}$ mixing-time, whereas under transitivity the same is true for the intersection time of two independent such BRWs. Using the same methodology, we show that for a sequence of reversible Ma…
We study two types of Metropolis-Hastings (MH) reversiblizations for non-reversible Markov chains with Markov kernel $P$. While the first type is the classical Metropolised version of $P$, we introduce a new self-adjoint kernel which captures the opposite transition effect of the first type, that we call the second MH kernel. We investigate the spectral relationship between $P$ and the two MH kernels. Along the way, we state a version of Weyl's inequality for the spectral gap of $P$ (and hence its additive reversiblization), as well as an expansion of $P$. Both results are expressed in terms o…
The asymptotic variance is an important criterion to evaluate the performance of Markov chains, especially for the central limit theorems. We give the variational formulas for the asymptotic variance of discrete-time (non-reversible) Markov chains on general state space. The variational formulas provide many applications, extending the classical Peskun's comparison theorem to non-reversible Markov chains, and obtaining several comparison theorems between Markov chains with various perturbations.
This paper presents an elementary proof of stochastic stability of a discrete-time reversible Markov chain starting from a Foster-Lyapunov drift condition. Besides its relative simplicity, there are two salient features of the proof: (i) it relies entirely on functional-analytic non-probabilistic arguments; and (ii) it makes explicit the connection between a Foster-Lyapunov function and Poincaré inequality. The proof is used to derive an explicit bound for the spectral gap. An extension to the non-reversible case is also presented.
Evaluating the degree of partisan districting (Gerrymandering) in a statistical framework typically requires an ensemble of districting plans which are drawn from a prescribed probability distribution that adheres to a realistic and non-partisan criteria. In this article we introduce novel non-reversible Markov chain Monte-Carlo (MCMC) methods for the sampling of such districting plans which have improved mixing properties in comparison to previously used (reversible) MCMC algorithms. In doing so we extend the current framework for construction of non-reversible Markov chains on discrete sampl…
A Markov decision problem is called reversible if the stationary controlled Markov chain is reversible under every stationary Markovian strategy. A natural application in which such problems arise is in the control of Metropolis-Hastings type dynamics. We characterize all discrete time reversible Markov decision processes with finite state and actions spaces. We show that policy iteration algorithm for finding an optimal policy can be significantly simplified Markov decision problems of this type. We also highlight the relation between the finite time evolution of the accrual of reward and the…
Capacity is an important quantity in potential theory and in the study of Markov processes. We give equivalent conditions between the capacity, the mean exit time, and the Green function for non-reversible diffusions.
We consider the problem of testing the identity of a reversible Markov chain against a reference from a single trajectory of observations. Employing the recently introduced notion of a lumping-congruent Markov embedding, we show that, at least in a mildly restricted setting, testing identity to a reversible chain reduces to testing to a symmetric chain over a larger state space and recover state-of-the-art sample complexity for the problem.
This article considers a class of metastable non-reversible diffusion processes whose invariant measure is a Gibbs measure associated with a Morse potential. In a companion paper [32], we proved the Eyring-Kramers formula for the corresponding class of metastable diffusion processes. In this article, we further develop this result by proving that a suitably time-rescaled metastable diffusion process converges to a Markov chain on the deepest metastable valleys. This article is also an extension of [45], which considered the same problem for metastable reversible diffusion processes. Our proof …
Cutoff for mixtures of permuted Markov chains: reversible case
Département d’Informatique de l’ENS, École Normale Supérieure（巴黎高等师范学院计算机科学系）；CNRS（法国国家科学研究中心）；PSL Research University（巴黎文理研究大学）；INRIA Paris（法国国家信息与自动化研究所巴黎分部）；CNRS, I2M, Aix-Marseille Université（法国国家科学研究中心、马赛数学研究所、艾克斯-马赛大学）
We prove a version of McDiarmid's bounded differences inequality for Markov chains, with constants proportional to the mixing time of the chain. We also show variance bounds and Bernstein-type inequalities for empirical averages of Markov chains. In the case of non-reversible chains, we introduce a new quantity called the "pseudo spectral gap", and show that it plays a similar role for non-reversible chains as the spectral gap plays for reversible chains. Our techniques for proving these results are based on a coupling construction of Katalin Marton, and on spectral techniques due to Pascal Le…
