This paper develops an optimal Chernoff type bound for the probabilities of large deviations of sums $\sum_{k=1}^n f (X_k)$ where $f$ is a real-valued function and $(X_k)_{k \in \mathbb{Z}_{\ge 0}}$ is a finite state Markov chain with an arbitrary initial distribution and an irreducible transition probability matrix satisfying a mild assumption on its positivity pattern, related to the function $f$ being considered. The novelty lies in this being a non-asymptotic finite sample bound. Further, our bound is optimal in the large deviations sense, attaining a constant prefactor and an exponential decay with the optimal large deviations rate. Moreover, through a Pinsker type inequality and a Hoeffding type lemma, we are able to loosen up our Chernoff type bound to a Hoeffding type bound and reveal the sub-Gaussian nature of the sums. Finally, under the same mild assumption on the positivity pattern of the transition probability matrix, we prove a uniform multiplicative ergodic theorem for the exponential family of tilted transition probability matrices corresponding to $f$.
Chernoff-Hoeffding Bounds for Markov Chains: Generalized and Simplified
基于链的标准L1（变化距离）混合时间，证明了一般不可逆有限状态马尔可夫链的第一Chernoff-Hoeffding界。具体地，考虑遍历马尔可夫链M和M的状态空间[n]上的权函数f：[n] -> [0，1]，其中平均值mu = E_{v <- pi} [f（v）]，其中pi是M的平稳分布。t步随机游动（v_1，.，v_t）具有期望的总权重E[X] = mu t，其中X = sum_{i=1} ^t f（v_i）。设T为M的L_1混合时间。我们表明，X偏离其平均值的概率由一个乘法因子δ，即，PR [|X - mu t|>= delta mu t ]，对于0 <= delta <= 1，至多为exp（-Omega（delta ^2 mu t / T）），对于delta > 1，至多为exp（-Omega（delta mu t / T））。事实上，即使i在[t]中的权函数f_i是不同的，只要它们都具有相同的平均值μ，界限也是成立的。基于M的谱展开式λ，我们也得到了M-Hoeffding界的一个简化证明，λ是M的第二大特征值（以绝对值表示）的平方根，其中M是M的时间反演马尔可夫链.我们证明了概率Pr [|X - mu t|对于0 <= delta <= 1，>= delta mu t ]至多为exp（-Omega（delta（1-lambda）mu t）），对于delta > 1，至多为ex…
In this article, we establish Hoeffding's inequality for bounded Lipschitz functions of a class of not necessarily irreducible Markov models. The result complements the existing literature on this topic where Hoeffding's inequality for bounded measurable functions of a class of irreducible Markov models has been considered. Our approach is based on the assumption of uniform ergodicity of the underlying Markov model in $L^1$-Wasserstein space.
We consider Markov chain with spectral gap in $L^2$ space. Assume that $f$ is a bounded function. Then the probabilities of large deviations of average along trajectory satisfy Hoeffding's-type inequalities. These bounds depend only on the stationary mean, spectral gap and the end-points of support of $f$.
The Markov, Chebyshev, and Chernoff inequalities are some of the most widely used methods for bounding the tail probabilities of random variables. In all three cases, the bounds are tight in the sense that there exists easy examples where the inequalities become equality. Here we will show that through a simple smoothing using auxiliary randomness, that each of the three bounds can be cut in half. In many common cases, the halving can be achieved without the need for the auxiliary randomness.
We prove deviation bounds for the random variable $\sum_{i=1}^{n} f_i(Y_i)$ in which $\{Y_i\}_{i=1}^{\infty}$ is a Markov chain with stationary distribution and state space $[N]$, and $f_i: [N] \rightarrow [-a_i, a_i]$. Our bound improves upon previously known bounds in that the dependence is on $\sqrt{a_1^2+\cdots+a_n^2}$ rather than $\max_{i}\{a_i\}\sqrt{n}.$ We also prove deviation bounds for certain types of sums of vector--valued random variables obtained from a Markov chain in a similar manner. One application includes bounding the expected value of the Schatten $\infty$-norm of a random…
We prove a Chernoff-type bound for sums of matrix-valued random variables sampled via a regular (aperiodic and irreducible) finite Markov chain. Specially, consider a random walk on a regular Markov chain and a Hermitian matrix-valued function on its state space. Our result gives exponentially decreasing bounds on the tail distributions of the extreme eigenvalues of the sample mean matrix. Our proof is based on the matrix expander (regular undirected graph) Chernoff bound [Garg et al. STOC '18] and scalar Chernoff-Hoeffding bounds for Markov chains [Chung et al. STACS '12]. Our matrix Chernoff…
This work prepares new probability bounds for sums of random, independent, Hermitian tensors. These probability bounds characterize large-deviation behavior of the extreme eigenvalue of the sums of random tensors. We extend Lapalace transform method and Lieb's concavity theorem from matrices to tensors, and apply these tools to generalize the classical bounds associated with the names Chernoff, Bennett, and Bernstein from the scalar to the tensor setting. Tail bounds for the norm of a sum of random rectangular tensors are also derived from corollaries of random Hermitian tensors cases. The pro…
For every given real value of the ratio $μ:=A_X/G_X>1$ of the arithmetic and geometric means of a positive random variable $X$ and every real $v>0$, exact upper bounds on the right- and left-tail probabilities $\mathsf{P}(X/G_X\ge v)$ and $\mathsf{P}(X/G_X\le v)$ are obtained, in terms of $μ$ and $v$. In particular, these bounds imply that $X/G_X\to1$ in probability as $A_X/G_X\downarrow1$. Such a result may be viewed as a converse to a reverse Jensen inequality for the strictly concave function $f=\ln$, whereas the well-known Cantelli and Chebyshev inequalities may be viewed as converses to a…
Slow mixing is the central hurdle when working with Markov chains, especially those used for Monte Carlo approximations (MCMC). In many applications, it is only of interest to estimate the stationary expectations of a small set of functions, and so the usual definition of mixing based on total variation convergence may be too conservative. Accordingly, we introduce function-specific analogs of mixing times and spectral gaps, and use them to prove Hoeffding-like function-specific concentration inequalities. These results show that it is possible for empirical expectations of functions to concen…
This paper is the Part II of a serious work about T product tensors focusing at establishing new probability bounds for sums of random, independent, T product tensors. These probability bounds characterize large deviation behavior of the extreme eigenvalue of the sums of random T product tensors. We apply Lapalace transform method and Lieb concavity theorem for T product tensors obtained from our Part I paper, and apply these tools to generalize the classical bounds associated with the names Chernoff, and Bernstein from the scalar to the T product tensor setting. Tail bounds for the norm of a …
Let $\{W_t\}_{t=1}^{\infty}$ be a finite state stationary Markov chain, and suppose that $f$ is a real-valued function on the state space. If $f$ is bounded, then Gillman's expander Chernoff bound (1993) provides concentration estimates for the random variable $f(W_1)+\cdots+f(W_n)$ that depend on the spectral gap of the Markov chain and the assumed bound on $f$. Here we obtain analogous inequalities assuming only that the $q$'th moment of $f$ is bounded for some $q \geq 2$. Our proof relies on reasoning that differs substantially from the proofs of Gillman's theorem that are available in the …
University of Florida（佛罗里达大学）；Inria（法国国家信息与自动化研究所）；École Normale Supérieure（巴黎高等师范学院）；CNRS（法国国家科学研究中心）；PSL Research University（巴黎文理研究大学）
This paper concerns error bounds for recursive equations subject to Markovian disturbances. Motivating examples abound within the fields of Markov chain Monte Carlo (MCMC) and Reinforcement Learning (RL), and many of these algorithms can be interpreted as special cases of stochastic approximation (SA). It is argued that it is not possible in general to obtain a Hoeffding bound on the error sequence, even when the underlying Markov chain is reversible and geometrically ergodic, such as the M/M/1 queue. This is motivation for the focus on mean square error bounds for parameter estimates. It is s…
We consider the problem of finding the optimal upper bound for the tail probability of a sum of $k$ nonnegative, independent and identically distributed random variables with given mean $x$. For $k=1$ the answer is given by Markov's inequality and for $k=2$ the solution was found by Hoeffding and Shrikhande in 1955. We solve the problem for $k=3$ as well as for general $k$ and $x\leq1/(2k-1)$ by showing that it follows from the fractional version of an extremal graph theory problem of Erdős on matchings in hypergraphs.
