Carlos A. Leon, Francois Perron
We build optimal exponential bounds for the probabilities of large deviations of sums \sum_{k=1}^nf(X_k) where (X_k) is a finite reversible Markov chain and f is an arbitrary bounded function. These bounds depend only on the stationary mean E_πf, the end-points of the support of f, the sample size n and the second largest eigenvalue λof the transition matrix.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
继续探索
In this article, we establish Hoeffding's inequality for bounded Lipschitz functions of a class of not necessarily irreducible Markov models. The result complements the existing literature on this topic where Hoeffding's inequality for bounded measurable functions of a class of irreducible Markov models has been considered. Our approach is based on the assumption of uniform ergodicity of the underlying Markov model in $L^1$-Wasserstein space.
We consider Markov chain with spectral gap in $L^2$ space. Assume that $f$ is a bounded function. Then the probabilities of large deviations of average along trajectory satisfy Hoeffding's-type inequalities. These bounds depend only on the stationary mean, spectral gap and the end-points of support of $f$.
We prove deviation bounds for the random variable $\sum_{i=1}^{n} f_i(Y_i)$ in which $\{Y_i\}_{i=1}^{\infty}$ is a Markov chain with stationary distribution and state space $[N]$, and $f_i: [N] \rightarrow [-a_i, a_i]$. Our bound improves upon previously known bounds in that the dependence is on $\sqrt{a_1^2+\cdots+a_n^2}$ rather than $\max_{i}\{a_i\}\sqrt{n}.$ We also prove deviation bounds for certain types of sums of vector--valued random variables obtained from a Markov chain in a similar manner. One application includes bounding the expected value of the Schatten $\infty$-norm of a random…
A short operator proof of Hoeffding inequalities for Markov chains
本文用两个关键观察（算子分解与秩一扰动）给出马尔可夫链加性泛函及随机矩阵Hoeffding不等式的简短算子证明，恢复并改进现有结果。
Hoeffding's Inequality for Markov Chains under Generalized Concentrability Condition
本文提出基于积分概率度量的广义可集中性条件下马尔可夫链Hoeffding不等式框架，扩展了传统适用范围，并验证其在机器学习多类非渐近分析中的效用。
We prove the first Chernoff-Hoeffding bounds for general nonreversible finite-state Markov chains based on the standard L_1 (variation distance) mixing-time of the chain. Specifically, consider an ergodic Markov chain M and a weight function f: [n] -> [0,1] on the state space [n] of M with mean mu = E_{v <- pi}[f(v)], where pi is the stationary distribution of M. A t-step random walk (v_1,...,v_t) on M starting from the stationary distribution pi has expected total weight E[X] = mu t, where X = sum_{i=1}^t f(v_i). Let T be the L_1 mixing-time of M. We show that the probability of X deviating f…
This paper develops an optimal Chernoff type bound for the probabilities of large deviations of sums $\sum_{k=1}^n f (X_k)$ where $f$ is a real-valued function and $(X_k)_{k \in \mathbb{Z}_{\ge 0}}$ is a finite state Markov chain with an arbitrary initial distribution and an irreducible transition probability matrix satisfying a mild assumption on its positivity pattern, related to the function $f$ being considered. The novelty lies in this being a non-asymptotic finite sample bound. Further, our bound is optimal in the large deviations sense, attaining a constant prefactor and an exponential …
Hoeffding's inequality for continuous-time Markov chains
本文针对可数状态空间上不可约且正常返的连续时间马尔可夫链（CTMC），利用骨架链和增广截断逼近技术推导出专门的 Hoeffding 不等式，将其从离散时间推广至连续时间，并探讨了在一般状态空间跳跃过程上的应用。
Embedding of reversible Markov matrices
本文研究了可逆马尔可夫矩阵嵌入时间齐次马尔可夫半群的问题，探讨了弱可逆矩阵和具有负特征值的可逆矩阵的嵌入条件。
We propose some backward-forward martingale decompositions for functions of reversible Markov chains. These decompositions are used to prove the functional CLT for reversible Markov chains with asymptotically linear variance of partial sums. We also provide a proof of the equivalence between asymptotic linearity of the variance and convergence of the integral of $1/(1-t)$ with respect to the associated spectral measure $ρ$. We also study the asymptotic behavior of linear processes having as innovations mean zero square integrable functions of stationary reversible Markov chains. We apply this …
In this paper we study the almost sure central limit theorem started from a point for additive functionals of a stationary and ergodic Markov chain via a martingale approximation in the almost sure sense. As a consequence we derive the quenched CLT for additive functionals or reversible Markov chains.
The theory of $L^2$-spectral gaps for reversible Markov chains has been studied by many authors. In this paper we consider positive recurrent general state space Markov chains with stationary transition probabilities. Replacing the assumption of reversibility by a less strong one, we still obtain a simple necessary and sufficient condition for the spectral gap property of the associated Markov operator in terms of isoperimetric constant. Moreover, we define a new sequence of isoperimetric constants which provides a necessary and sufficient condition for the existence of a spectral gap in a ver…
In this paper we investigate the kernel estimator of the density for a stationary reversible Markov chain. The proofs are based on a new central limit theorem for a triangular array of reversible Markov chains obtained under conditions imposed to covariances, which has interest in itself.
Reversible Markov kernels and involutions on product spaces
该研究探讨乘积空间上保独立性对合与可逆马尔可夫核的关系，提出可测函数的对合增广构造，将文献中多类示例归为其特例，并证明带反射壁随机游走的马尔可夫核生成的IP性质可刻画几何型分布。
It is well known that for a strictly stationary, reversible, Harris recurrent Markov chain, the $ρ$-mixing condition is equivalent to geometric ergodicity and to a "spectral gap" condition. In this note, it will be shown with an example that for that class of Markov chains, the "interlaced" variant of the $ρ$-mixing condition fails to be equivalent to those conditions.
We obtain universal estimates on the convergence to equilibrium and the times of coupling for continuous time irreducible reversible finite-state Markov chains, both in the total variation and in the L^2 norms. The estimates in total variation norm are obtained using a novel identity relating the convergence to equilibrium of a reversible Markov chain to the increase in the entropy of its one-dimensional distributions. In addition, we propose a universal way of defining the ultrametric partition structure on the state space of such Markov chains. Finally, for chains reversible with respect to …
Concentration Inequalities for Sums of Markov Dependent Random Matrices
本文针对一般状态空间上的时间依赖矩阵值函数，利用非交换矩母函数和谱方法，推导出 Markov 链随机矩阵和最大特征值的 Hoeffding 与 Bernstein 型集中不等式，推广了已有结果并建立了与动力系统及 Banach 空间理论的联系。
The embedding problem for Markov chains is a famous problem in probability theory and only partial results are available up till now. In this paper, we propose a variant of the embedding problem called the reversible embedding problem which has a deep physical and biochemical background and provide a complete solution to this new problem. We prove that the reversible embedding of a stochastic matrix, if it exists, must be unique. Moreover, we obtain the sufficient and necessary conditions for the existence of the reversible embedding and provide an effective method to compute the reversible em…
It has been well known for some time that for strictly stationary Markov chains that are ``reversible'', that special symmetry provides special extra features in the mathematical theory. This paper here is primarily a purely expository review of some of the basic features of that special theory. The mathematical techniques employed in this review are relatively gentle, involving only some basic measure-theoretic probability theory.
We extend the Dirichlet principle to non-reversible Markov processes on countable state spaces. We present two variational formulas for the solution of the Poisson equation or, equivalently, for the capacity between two disjoint sets. As an application we prove a some recurrence theorems. In particular, we show the recurrence of two-dimensional cycle random walks under a second moment condition on the winding numbers.
In this paper, we develop an in-depth analysis of non-reversible Markov chains on denumerable state space from a similarity orbit perspective. In particular, we study the class of Markov chains whose transition kernel is in the similarity orbit of a normal transition kernel, such as the one of birth-death chains or reversible Markov chains. We start by identifying a set of sufficient conditions for a Markov chain to belong to the similarity orbit of a birth-death one. As by-products, we obtain a spectral representation in terms of non-self-adjoint resolutions of identity in the sense of Dunfor…
A class of examples is constructed to show that for strictly stationary Markov chains that are reversible, the simultaneous mixing rates for the $ρ$-mixing and strong mixing ($α$-mixing) conditions can be fairly arbitrary, within certain unavoidable tight restrictions. The examples constructed here have the added property that the mixing rate for the absolute regularity ($β$-mixing) condition is within a constant factor of that for strong mixing.
We obtain necessary and sufficient conditions for the regular variation of the variance of partial sums of functionals of discrete and continuous-time stationary Markov processes with normal transition operators. We also construct a class of Metropolis-Hastings algorithms which satisfy a central limit theorem and invariance principle when the variance is not linear in $n$.
In this paper, we investigate the functional central limit theorem for stochastic processes associated to partial sums of additive functionals of reversible Markov chains with general spate space, under the normalization standard deviation of partial sums. For this case, we show that the functional central limit theorem is equivalent to the fact that the variance of partial sums is regularly varying with exponent 1 and the partial sums satisfy the CLT. It is also equivalent to the conditional CLT.
A variational formula for the asymptotic variance of general Markov processes is obtained. As application, we get a upper bound of the mean exit time of reversible Markov processes, and some comparison theorems between the reversible and non-reversible diffusion processes.
In this paper we study the spectral properties of Markov-operator on $L^{2}$-spaces. Lawler and Sokal (Trans. Amer. Math. Soc., 1988, 309, pp. 557-580) used isoperimetric constants for discrete and continuous time Markov chains to obtain a spectral gap at 1. For time discrete Markov chains this does not exclude periodic behavior. We define a new constant measuring the distance from periodicity and give necessary and sufficient conditions for the existence of a global spectral gap in terms of this constant.
We present a Markov chain example where non-reversibility and an added edge jointly improve mixing time: when a random edge is added to a cycle of $n$ vertices and a Markov chain with a drift is introduced, we get mixing time of $O(n^{3/2})$ with probability bounded away from 0. If only one of the two modifications were performed, the mixing time would stay $Ω(n^2)$.
On a boundary of the central limit theorem for strictly stationary, reversible Markov chains
本文证明可逆严格平稳马尔可夫链中心极限定理成立的边界混合速率恰为指数速率，并构造反例表明混合速率任意接近指数时定理可失效。
We prove a complete class theorem that characterizes \emph{all} stationary time reversible Markov processes whose finite dimensional marginal distributions (of all orders) are infinitely divisible. Aside from two degenerate cases (iid and constant), in both discrete and continuous time every such process with full support is a branching process with Poisson or Negative Binomial marginal univariate distributions and a specific bivariate distribution at pairs of times. As a corollary, we prove that every nondegenerate stationary integer valued processes constructed by the Markov thinning process…
On Central Limit Theorems for Additive Functionals of Reversible Ergodic Markov Processes
本文证明了Bhattacharya的一般定理在时间可逆情况下的推论，推导了Kipnis-Varadhan功能中心极限定理的核心方法与主要贡献。
The large deviations principle for the empirical measure for both continuous and discrete time Markov processes is well known. Various expressions are available for the rate function, but these expressions are usually as the solution to a variational problem, and in this sense not explicit. An interesting class of continuous time, reversible processes was identified in the original work of Donsker and Varadhan for which an explicit expression is possible. While this class includes many (reversible) processes of interest, it excludes the case of continuous time pure jump processes, such as a re…
By proving a local limit theorem for higher-order transitions, we determine the time required for necklace chains to be close to stationarity. Because necklace chains, built by arranging identical smaller chains around a directed cycle, are not reversible, have little symmetry, do not have uniform stationary distributions, and can be nearly periodic, prior general bounds on rates of convergence of Markov chains either do not apply or give poor bounds. Necklace chains can serve as test cases for future techniques for bounding rates of convergence.
Markov chain Monte Carlo(MCMC) is a popular approach to sample from high dimensional distributions, and the asymptotic variance is a commonly used criterion to evaluate the performance. While most popular MCMC algorithms are reversible, there is a growing literature on the development and analyses of nonreversible MCMC. Chen and Hwang(2013) showed that a reversible MCMC can be improved by adding an antisymmetric perturbation. They also raised a conjecture that it can not be improved if there is no cycle in the corresponding graph. In this paper, we present a rigorous proof of this conjecture. …
Bounds on Spectral Gaps for Non-Reversible Markov Chains with Applications to Temporal Difference Learning
本文为分析时序差分算法，通过Dirichlet谱间隙界定非可逆马尔可夫链的特征值，推广了稳定性充分条件，并给出离散时间线性高斯系统的显式界。
We take on a Random Matrix theory viewpoint to study the spectrum of certain reversible Markov chains in random environment. As the number of states tends to infinity, we consider the global behavior of the spectrum, and the local behavior at the edge, including the so called spectral gap. Results are obtained for two simple models with distinct limiting features. The first model is built on the complete graph while the second is a birth-and-death dynamics. Both models give rise to random matrices with non independent entries.
In this paper we study the asymptotic behavior of linear processes having as innovations mean zero, square integrable functions of stationary reversible Markov chains. In doing so we shall preserve the generality of coefficients assuming only that they are square summable. In this way we include in our study the long range dependence case. The only assumption imposed on the innovations is the absolute summability of their covariances. Besides the central limit theorem we also study the convergence to fractional Brownian motion. The proofs are based on general results for linear processes with …
In this article, we considers reversible Markov chains of which $L^2$-distances can be expressed in terms of Laplace transforms. The cutoff of Laplace transforms was first discussed by Chen and Saloff-Coste in [8], while we provide here a completely different pathway to analyze the $L^2$-distance. Consequently, we obtain several considerably simplified criteria and this allows us to proceed advanced theoretical studies, including the comparison of cutoffs between discrete time lazy chains and continuous time chains. For an illustration, we consider product chains, a rather complicated model wh…
In this paper we prove a version of the Berman-Konsowa principle for reversible Markov jump processes on Polish spaces. The Berman-Konsowa principle provides a variational formula for the capacity of a pair of disjoint measurable sets. There are two versions, one involving a class of probability measures for random finite paths from one set to the other, the other involving a class of finite unit flows from one set to the other. The Berman-Konsowa principle complements the Dirichlet principle and the Thomson principle, and turns out to be especially useful for obtaining sharp estimates on cros…
We introduce the notion of order of magnitude reversibility (OM-reversibility) in Markov chains that are parametrized by a positive parameter $\ep$. OM-reversibility is a weaker condition than reversibility, and requires only the knowledge of order of magnitude of the transition probabilities. For an irreducible, OM-reversible Markov chain on a finite state space, we prove that the stationary distribution satisfies order of magnitude detailed balance (analog of detailed balance in reversible Markov chains). The result characterizes the states with positive probability in the limit of the stati…
A sequence of Markov chains is said to exhibit (total variation) cutoff if the convergence to stationarity in total variation distance is abrupt. We consider reversible lazy chains. We prove a necessary and sufficient condition for the occurrence of the cutoff phenomena in terms of concentration of hitting time of "worst" (in some sense) sets of stationary measure at least $α$, for some $α\in (0,1)$. We also give general bounds on the total variation distance of a reversible chain at time $t$ in terms of the probability that some "worst" set of stationary measure at least $α$ was not hit by ti…
Reversible Imprecise Markov Chains
本文为可逆不精确马尔可夫链建立理论框架，引入基于两步联合分布矩阵信任集的对称表示，将可逆性简化为矩阵对称性，并用于计算路径依赖泛函的期望。
It is shown that transient graphs for the simple random walk do not admit a nearest neighbor transient Markov chain (not necessarily a reversible one), that crosses all edges with positive probability, while there is such chain for the square grid $\mathbb{Z}^2$. In particular, the $d$-dimensional grid $\mathbb{Z}^d$ admits such a Markov chain only when $d=2$. For $d=2$ we present a relevant example due to Gady Kozma, while the general statement for transient graphs is obtained by proving that for every transient irreducible Markov chain on a countable state space, which admits a stationary me…
Let $S_N$ be the sum of vector-valued functions defined on a finite Markov chain. An analogue of the Bernstein--Hoeffding inequality is derived for the probability of large deviations of $S_N$ and relates the probability to the spectral gap of the Markov chain. Examples suggest that this inequality is better than alternative inequalities if the chain has a sufficiently large spectral gap and the function is high-dimensional.
We develop Markov chain mixing time estimates for a class of Markov chains with restricted transitions. We assume transitions may occur along a cycle of $n$ nodes and on $n^γ$ additional edges, where $γ< 1$. We find that the mixing times of reversible Markov chains properly interpolate between the mixing times of the cycle with no added edges and of the cycle with $cn$ added edges (which is in turn a Small World Network model). In the case of non-reversible Markov-chains, a considerable gap remains between lower and upper bounds, but simulations give hope to experience a significant speedup co…
Consider a sequence of continuous-time Markov chains $(X^{(n)}_t:t\ge 0)$ evolving on a fixed finite state space $V$. Let $I_n$ be the level two large deviations rate functional for $X^{(n)}_t$, as $t\to\infty$. Under a hypothesis on the jump rates, we prove that $I_n$ can be written as $I_n = I^{(0)} \,+\, \sum_{1\le p\le q} (1/θ^{(p)}_n) \, I^{(p)}$ for some rate functionals $I^{(p)}$. The weights $θ^{(p)}_n$ correspond to the time-scales at which the sequence of Markov chains $X^{(n)}_t$ exhibit a metastable behavior, and the zero level sets of the rate functionals $I^{(p)}$ identify the me…
The classical Metropolis-Hastings (MH) algorithm can be extended to generate non-reversible Markov chains. This is achieved by means of a modification of the acceptance probability, using the notion of vorticity matrix. The resulting Markov chain is non-reversible. Results from the literature on asymptotic variance, large deviations theory and mixing time are mentioned, and in the case of a large deviations result, adapted, to explain how non-reversible Markov chains have favorable properties in these respects. We provide an application of NRMH in a continuous setting by developing the necessa…
Random sampling of large Markov matrices with a tunable spectral gap, a nonuniform stationary distribution, and a nondegenerate limiting empirical spectral distribution (ESD) is useful. Fix $c>0$ and $p>0$. Let $A_n$ be the adjacency matrix of a random graph following $\mathrm{G}(n, p/n)$, known as the Erdős-Rényi distribution. Add $c/n$ to each entry of $A_n$ and then normalize its rows. It is shown that the resulting Markov matrix has the desired properties. Its ESD weakly converges in probability to a symmetric nondegenerate distribution, and its extremal eigenvalues, other than 1, fall in …
We present results relating mixing times to the intersection time of branching random walk (BRW) in which the logarithm of the expected number of particles grows at rate of the spectral-gap $\mathrm{gap}$ . This is a finite state space analog of a critical branching process. Namely, we show that the maximal expected hitting time of a state by such a BRW is up to a universal constant larger than the $L_{\infty}$ mixing-time, whereas under transitivity the same is true for the intersection time of two independent such BRWs. Using the same methodology, we show that for a sequence of reversible Ma…
Approximations of Geometrically Ergodic Reversible Markov Chains
本文提出基于希尔伯特空间$ L_2(π) $的工具，用于评估几何性渐近可逆马尔可夫链近似的准确性和效率，扩展了Johndrow等人[18]的结果，并应用于多种近似MCMC算法。
For a Markov process associated with a diffusion type Dirichlet form an upper bound is shown for the law of the finite dimensional distributions of the process. Under some more assumptions on the underlaying space this is also shown for the law of the Markov process itself. In the last section we want to give an application to the Wasserstein diffusion.
We prove the metastable behavior of reversible Markov processes on finite state spaces under minimal conditions on the jump rates. To illustrate the result we deduce the metastable behavior of the Ising model with a small magnetic field at very low temperature.
We study two types of Metropolis-Hastings (MH) reversiblizations for non-reversible Markov chains with Markov kernel $P$. While the first type is the classical Metropolised version of $P$, we introduce a new self-adjoint kernel which captures the opposite transition effect of the first type, that we call the second MH kernel. We investigate the spectral relationship between $P$ and the two MH kernels. Along the way, we state a version of Weyl's inequality for the spectral gap of $P$ (and hence its additive reversiblization), as well as an expansion of $P$. Both results are expressed in terms o…
Continuity for Limit Profiles of Reversible Markov Chains
本文证明在涉及转移矩阵谱与截止窗口的可计算条件下，具有全变差截止的可逆马尔可夫链序列的极限轮廓是连续函数。
The asymptotic variance is an important criterion to evaluate the performance of Markov chains, especially for the central limit theorems. We give the variational formulas for the asymptotic variance of discrete-time (non-reversible) Markov chains on general state space. The variational formulas provide many applications, extending the classical Peskun's comparison theorem to non-reversible Markov chains, and obtaining several comparison theorems between Markov chains with various perturbations.
Comparing the Efficiency of General State Space Reversible MCMC Algorithms
本文在一般状态空间上研究可逆MCMC算法的效率比较，证明效率支配等价于算子$Q-P$在$L^2(π)$上正定，并据此推广了分量核组合的效率支配结果。
Capacity is an important quantity in potential theory and in the study of Markov processes. We give equivalent conditions between the capacity, the mean exit time, and the Green function for non-reversible diffusions.
This paper presents an elementary proof of stochastic stability of a discrete-time reversible Markov chain starting from a Foster-Lyapunov drift condition. Besides its relative simplicity, there are two salient features of the proof: (i) it relies entirely on functional-analytic non-probabilistic arguments; and (ii) it makes explicit the connection between a Foster-Lyapunov function and Poincaré inequality. The proof is used to derive an explicit bound for the spectral gap. An extension to the non-reversible case is also presented.
We provide conditions that classify cover times for sequences of random walks on random graphs into two types: One type (Type 1) is the class of cover times that are of the order of the maximal hitting times scaled by the logarithm of the size of vertex sets. The other type (Type 2) is the class of cover times that are of the order of the maximal hitting times. The conditions are described by some parameters determined by the underlying graphs: the volumes, the diameters with respect to the resistance metric, the coverings or packings by balls in the resistance metric. We apply the conditions …
This paper originally showed a lower bound on mixing time for a non-reversible Markov chain in terms of its largest non-trivial eigenvalue, and used this to re-derive some generalizations of results of Fan Chung. However, the paper has been withdrawn because I have been made aware that the lower bound was previously known.
Evaluating the degree of partisan districting (Gerrymandering) in a statistical framework typically requires an ensemble of districting plans which are drawn from a prescribed probability distribution that adheres to a realistic and non-partisan criteria. In this article we introduce novel non-reversible Markov chain Monte-Carlo (MCMC) methods for the sampling of such districting plans which have improved mixing properties in comparison to previously used (reversible) MCMC algorithms. In doing so we extend the current framework for construction of non-reversible Markov chains on discrete sampl…
扫码添加微信好友，提出您的宝贵建议 👇
💡 备注请填写：网站反馈
发表机构由 AI 从论文文本抽取，如发现错误，欢迎反馈修正建议。