In a celebrated work by Hoeffding [J. Amer. Statist. Assoc. 58 (1963) 13-30], several inequalities for tail probabilities of sums M_n=X_1+... +X_n of bounded independent random variables X_j were proved. These inequalities had a considerable impact on the development of probability and statistics, and remained unimproved until 1995 when Talagrand [Inst. Hautes Etudes Sci. Publ. Math. 81 (1995a) 73-205] inserted certain missing factors in the bounds of two theorems. By similar factors, a third theorem was refined by Pinelis [Progress in Probability 43 (1998) 257-314] and refined (and extended) by me. In this article, I introduce a new type of inequality.
Namely, I show that P{M_n\geq x}\leq cP{S_n\geq x}, where c is an absolute constant and S_n=ε_1+... +ε_n is a sum of independent identically distributed Bernoulli random variables (a random variable is called Bernoulli if it assumes at most two values). The inequality holds for those x\in R where the survival function x\mapsto P{S_n\geq x} has a jump down. For the remaining x the inequality still holds provided that the function between the adjacent jump points is interpolated linearly or \log-linearly. If it is necessary, to estimate P{S_n\geq x} special bounds can be used for binomial probabilities. The results extend to martingales with bounded differences. It is apparent that Theorem 1.1 of this article is the most important.
We consider Markov chain with spectral gap in $L^2$ space. Assume that $f$ is a bounded function. Then the probabilities of large deviations of average along trajectory satisfy Hoeffding's-type inequalities. These bounds depend only on the stationary mean, spectral gap and the end-points of support of $f$.
We prove concentration inequalities for functions of independent random variables {under} sub-gaussian and sub-exponential conditions. The utility of the inequalities is demonstrated by an extension of the now classical method of Rademacher complexities to Lipschitz function classes and unbounded sub-exponential distribution.
We give an extension of Hoeffding's inequality to the case of supermartingales with differences bounded from above. Our inequality strengthens or extends the inequalities of Freedman, Bernstein, Prohorov, Bennett and Nagaev.
We obtain Azuma bounds for the probabilities of being away from the limit for a class of urn models. The method consists of relating the variables to certain linear combinations using eigenvectors of the replacement matrix, thus bringing in appropriate martingales. Some cases of repeated eigenvalues are also considered using Jordan vectors.
Beijing National Research Center for Information Science and Technology（北京信息科学与技术国家研究中心）；Tsinghua University（清华大学）
It is well known that Hoeffding's inequality has a lot of applications in the signal and information processing fields. How to improve Hoeffding's inequality and find the refinements of its applications have always attracted much attentions. An improvement of Hoeffding inequality was recently given by Hertz \cite{r1}. Eventhough such an improvement is not so big, it still can be used to update many known results with original Hoeffding's inequality, especially for Hoeffding-Azuma inequality for martingales. However, the results in original Hoeffding's inequality and its refinement one by Hertz…
We build optimal exponential bounds for the probabilities of large deviations of sums \sum_{k=1}^nf(X_k) where (X_k) is a finite reversible Markov chain and f is an arbitrary bounded function. These bounds depend only on the stationary mean E_πf, the end-points of the support of f, the sample size n and the second largest eigenvalue λof the transition matrix.
The Hoeffding-type-inequalities are obtained for the distribution tails of canonical (degenerate) U- and V-statistics of an arbitrary order based on samples from a stationary sequence of observations satisfying $ρ$-mixing.
Let $M_n= \fsu X1n$ be a sum of independent random variables such that $ X_k\leq 1$, $\E X_k =0$ and $\E X_k^2=\s_k^2$ for all $k$. Hoeffding 1963, Theorem 3, proved that $$¶{M_n \geq nt}\leq H^n(t,p),\quad H(t,p)= \bgl(1+qt/p\bgr)^{p +qt} \bgl({1-t}\bgr)^{q -qt}$$ with $$q=\ffrac 1{1+\s^2},\quad p=1-q, \quad \s^2 =\ffrac {\s_1^2+...+\s_n^2}n,\quad 0<t<1.$$ Bentkus 2004 improved Hoeffding's inequalities using binomial tails as upper bounds. Let $\ga_k =\E X_k^3/\s_k^3$ and $ \vk_k= \E X_k^4/\s_k^4$ stand for the skewness and kurtosis of $X_k$. In this paper we prove (improved) counterparts of …
We use a generalization of Hoeffding's inequality to show concentration results for the free energy of disordered pinning models, assuming only that the disorder has a finite exponential moment. We also prove some concentration inequalities for directed polymers in random environment, which we use to establish a large deviations results for the end position of the polymer under the polymer measure.
Hoeffding-type exponential inequalities are obtained for the distribution tails of canonical von Mises' statistics of arbitrary order based on samples from a stationary sequence of random variables satisfying the φ-mixing condition. The present paper weakens the restrictions on the coefficient φ() which are contained in the paper "Exponential inequalities for the distributions of canonical U- and V-statistics of dependent observations" by Borisov and Volodko (2009). At the same time, we correct the corresponding proof in this paper.
We provide a systematic approach to deal with the following problem. Let $X_1,\ldots,X_n$ be, possibly dependent, $[0,1]$-valued random variables. What is a sharp upper bound on the probability that their sum is significantly larger than their mean? In the case of independent random variables, a fundamental tool for bounding such probabilities is devised by Wassily Hoeffding. In this paper we consider analogues of Hoeffding's result for sums of dependent random variables for which we have certain information on their dependency structure. We prove a result that yields concentration inequalitie…
We investigate the properties of a discrete-time martingale $\{X_m\}_{m\in \mathbb Z_{\geq 0}}$, where all differences between adjacent random variables are limited to be not more than a constant as a promise. In this situation, it is known that the Azuma-Hoeffding inequality holds, which gives an upper bound of a probability for exceptional events. The inequality gives a simple form of the upper bound, and it has been utilized for many investigations. However, the inequality is not tight. We give an explicit expression of a tight upper bound, and we show that it and the bound obtained from th…
We establish concentration inequalities for Lipschitz functions of dependent random variables, whose dependencies are specified by forests. We also give concentration results for decomposable functions, improving Janson's Hoeffding-type inequality for the summation of graph-dependent bounded variables. These results extend McDiarmid's bounded difference inequality to the dependent cases.
Louisiana State University（路易斯安那州立大学）；Southern University and A&M College（南方大学及农工学院）
We derive simple concentration inequalities for bounded random vectors, which generalize Hoeffding's inequalities for bounded scalar random variables. As applications, we apply the general results to multinomial and Dirichlet distributions to obtain multivariate concentration inequalities.
This paper derives exponential concentration inequalities and polynomial moment inequalities for the spectral norm of a random matrix. The analysis requires a matrix extension of the scalar concentration theory developed by Sourav Chatterjee using Stein's method of exchangeable pairs. When applied to a sum of independent random matrices, this approach yields matrix generalizations of the classical inequalities due to Hoeffding, Bernstein, Khintchine and Rosenthal. The same technique delivers bounds for sums of dependent random matrices and more general matrix-valued functions of dependent rand…
In this paper necessary and sufficient conditions are presented for heat kernel upper bounds for random walks on weighted graphs. Several equivalent conditions are given in the form of isoperimetric inequalities.
For a real-valued measurable function $f$ and a nonnegative, nondecreasing function $ϕ$, we first obtain a Chebyshev type inequality which provides an upper bound for $\displaystyle ϕ(λ_{1}) μ(\{x \in Ω: f(x) \geq λ_{1} \}) + \sum_{k=2}^{n}\left(ϕ(λ_{k})- ϕ(λ_{k-1})\right) μ(\{x \in Ω: f(x) \geq λ_{k}\}) ,$ where $0 < λ_1 < λ_2 \cdots λ_n < \infty$. Using this, generalizations of a few concentration inequalities such as Markov, reverse Markov, Bienaymé-Chebyshev, Cantelli and Hoeffding inequalities are obtained.
We provide a general result for bounding the difference between point probabilities of integer supported distributions and the translated Poisson distribution, a convenient alternative to the discretized normal. We illustrate our theorem in the context of the Hoeffding combinatorial central limit theorem with integer valued summands, of the number of isolated vertices in an Erdős-Rényi random graph, and of the Curie-Weiss model of magnetism, where we provide optimal or near optimal rates of convergence in the local limit metric. In the Hoeffding example, even the discrete normal approximation …
The tails of the distribution of a mean zero, variance $σ^2$ random variable $Y$ satisfy concentration of measure inequalities of the form $\mathbb{P}(Y \ge t) \le \exp(-B(t))$ for $$ B(t)=\frac{t^2}{2( σ^2 + ct)} \quad \mbox{for $t \ge 0$, and} \quad B(t)=\frac{t}{c}\left( \log t - \log \log t - \frac{σ^2}{c}\right) \quad \mbox{for $t>e$} $$ whenever there exists a zero biased coupling of $Y$ bounded by $c$, under suitable conditions on the existence of the moment generating function of $Y$. These inequalities apply in cases where $Y$ is not a function of independent variables, such as for th…
We obtain a Bernstein type Gaussian concentration inequality for martingales. Our inequality improves the Azuma-Hoeffding inequality for moderate deviations $x$. Following the work of McDiarmid (1989), Talagrand (1996) and Boucheron, Lugosi and Massart (2000,2003), we show that our result can be applied to the concentration of random functions, Erdös-Rényi random graph, and maxima of empirical processes. Several interesting Gaussian concentration inequalities have been obtained.
Any infinite graph has site and bond percolation critical probabilities satisfying $p_c^{site}\geq p_c^{bond}$. The strict version of this inequality holds for many, but not all, infinite graphs. In this paper, the class of graphs for which the strict inequality holds is extended to a continuum percolation model. In Gilbert's graph with supercritical density on the Euclidean plane, there is almost surely a unique infinite connected component. We show that on this component $p_c^{site} > p_c^{bond}$. This also holds in higher dimensions.
Let $S$ and $X$ be independent random variables, assuming values in the set of non-negative integers, and suppose further that both $\mathbb{E}(S)$ and $\mathbb{E}(X)$ are integers satisfying $\mathbb{E}(S)\ge \mathbb{E}(X)$. We establish a sufficient condition for the tail probability $\mathbb{P}(S\ge \mathbb{E}(S))$ to be larger than $\mathbb{P}(S+X\ge \mathbb{E}(S+X))$. We also apply this result to sums of independent binomial and Poisson random variables.
This paper presents new probability inequalities for sums of independent, random, self-adjoint matrices. These results place simple and easily verifiable hypotheses on the summands, and they deliver strong conclusions about the large-deviation behavior of the maximum eigenvalue of the sum. Tail bounds for the norm of a sum of random rectangular matrices follow as an immediate corollary. The proof techniques also yield some information about matrix-valued martingales. In other words, this paper provides noncommutative generalizations of the classical bounds associated with the names Azuma, Benn…
In this paper, we investigate the ruin probabilities of non-homogeneous risk models. By employing martingale method, the Lundberg-type inequalities of ruin probabilities of non-homogeneous renewal risk models are obtained under weak assumptions. In addition, for the periodic and quasi-periodic risk models the adjustment coefficients of the Lundberg-type inequalities are obtained. Finally, examples are presented to show that estimations obtained in this paper are more accurate and the ruin probability in non-homogeneous risk models may be fast decreasing which is impossible for the case of homo…
In this paper we present various new inequalities for tail proabilities for distributions that are elements of the most improtant exponential families. These families include the Poisson distributions, the Gamma distributions, the binomial distributions, the negative binomial distributions and the inverse Gaussian distributions. All these exponential families have simple variance functions and the variance functions play an important role in the exposition. All the inequalities presented in this paper are formulated in terms of the signed log-likelihood. The inequalities are of a qualitative n…
We prove a new Bernstein-type inequality for the log-likelihood function of Bernoulli variables. In contrast to classical Bernstein's inequality and Hoeffding's inequality when applied to the log-likelihood, the new bound is independent of the parameters of the Bernoulli variables and therefore does not blow up as the parameters approach 0 or 1. The new inequality strengthens certain theoretical results on likelihood-based methods for community detection in networks and can be applied to other likelihood-based methods for binary data.
Upper exponential inequalities for the tail probabilities of the centered and normalized number of triangles in the Erdös-Rényi graph are obtained, where the probability of every edge is fixed. The result is formulated in terms of random fields.
Lundberg-type inequalities for ruin probabilities of non-homogeneous risk models are presented in this paper. By employing martingale method, the upper bounds of ruin probabilities are obtained for the general risk models under weak assumptions. In addition, several risk models, including the newly defined united risk model and quasi-periodic risk model with interest rate, are studied.
On any denumerable product of probability spaces, we construct a Malliavin gradient and then a divergence and a number operator. This yields a Dirichlet structure which can be shown to approach the usual structures for Poisson and Brownian processes. We obtain versions of almost all the classical functional inequalities in discrete settings which show that the Efron-Stein inequality can be interpreted as a Poincar{é} inequality or that Hoeffding decomposition of U-statistics can be interpreted as a chaos decomposition. We obtain a version of the Lyapounov central limit theorem for independent …
We show sharp bounds for probabilities of large deviations for sums of independent random variables satisfying Bernstein's condition. One such bound is very close to the tail of the standard Gaussian law in certain case; other bounds improve the inequalities of Bennett and Hoeffding by adding missing factors in the spirit of Talagrand (1995). We also complete Talagrand's inequality by giving a lower bound of the same form, leading to an equality. As a consequence, we obtain large deviation expansions similar to those of Cramér (1938), Bahadur-Rao (1960) and Sakhanenko (1991). We also show that…
We obtain some optimal inequalities on tail probabilities for sums of independent bounded random variables. Our main result completes an upper bound on tail probabilities due to Talagrand by giving a one-term asymptotic expansion for large deviations. This result can also be regarded as sharp large deviations of types of Cramér and Bahadur-Ranga Rao.
The Bernstein inequality is a tight upper bound on tail probabilities for independent random variables. Freedman extended the Bernstein inequality to martingales with differences bounded from above, and then Dzhaparidze and van Zanten generalized Freedman's result to non-bounded locally square integrable martingales. In this paper, we derive some Dzhaparidze-van Zanten type inequalities for self-normalized martingales with square integrable and non-square integrable differences.
We consider the supercritical finite-range random connection model where the points $x,y$ of a homogeneous planar Poisson process are connected with probability $f(|y-x|)$ for a given $f$. Performing percolation on the resulting graph, we show that the critical probabilities for site and bond percolation satisfy the strict inequality $p_c^{\rm site} > p_c^{\rm bond}$. We also show that reducing the connection function $f$ strictly increases the critical Poisson intensity. Finally, we deduce that performing a spreading transformation on $f$ (thereby allowing connections over greater distances b…
A survey is given of some Chernoff type bounds for the tail probabilities P(X-EX > a) and P(X-EX < a) when X is a random variable that can be written as a sum of indicator variables that are either independent or negatively related. Most bounds are previously known and some comparisons are made. This paper was written in 1994, but was never published because I had overlooked some existing papers containing some of the inequalities. Because of some recent interest in one of the inequalities, which does not seem to be published anywhere else, it has now been lightly edited and made available her…
We prove upper bounds on the transition probabilities of random walks with i.i.d. random conductances with a polynomial lower tail near $0$. We consider both constant and variable speed models. Our estimates are sharp. As a consequence, we derive local central limit theorems, parabolic Harnack inequalities and Gaussian bounds for the heat kernel. Some of the arguments are robust and applicable for random walks on general graphs. Such results are stated under a general setting.
CY University（CY大学）；University Paris 1 Panthéon-Sorbonne（巴黎第一大学先贤祠-索邦大学）
Concentration inequalities are widely used for analyzing machine learning algorithms. However, current concentration inequalities cannot be applied to some of the most popular deep neural networks, notably in natural language processing. This is mostly due to the non-causal nature of such involved data, in the sense that each data point depends on other neighbor data points. In this paper, a framework for modeling non-causal random fields is provided and a Hoeffding-type concentration inequality is obtained for this framework. The proof of this result relies on a local approximation of the non…
Recently, van den Berg and Jonasson gave the first substantial extension of the BK inequality for non-product measures: they proved that, for k-out-of-n measures, the probability that two increasing events occur disjointly is at most the product of the two individual probabilities. We show several other extensions and modifications of the BK inequality.In particular, we prove that the antiferromagnetic Ising Curie-Weiss model satisfies the BK inequality for all increasing events. We prove that this also holds for the Curie-Weiss model with three-body interactions under the so-called Negative L…
The goal of this article is two-fold: in a first part, we prove Azuma-Hoeffding type concentration inequalities around the drift for the displacement of non-elementary random walks on hyperbolic spaces. For a proper hyperbolic space $M$, we obtain explicit bounds that depend only on $M$, the size of support of the measure as in the classical case of sums of independent random variables, and on the norm of the driving probability measure in the left regular representation of the group of isometries. We obtain uniform bounds in the case of hyperbolic groups and effective bounds for simple linear…
We give a concentration inequality based on the premise that random variables take values within a particular region. The concentration inequality guarantees that, for any sequence of correlated random variables, the difference between the sum of conditional expectations and that of the observed values takes a small value with high probability when the expected values are evaluated under the condition that the past values are known. Our inequality outperforms other well-known inequalities, e.g. the Azuma-Hoeffding inequality, especially in terms of the convergence speed when the random variabl…
Small ball inequalities have been extensively studied in the setting of Gaussian processes and associated Banach or Hilbert spaces. In this paper, we focus on studying small ball probabilities for sums or differences of independent, identically distributed random elements taking values in very general sets. Depending on the setting--abelian or nonabelian groups, or vector spaces, or Banach spaces--we provide a collection of inequalities relating different small ball probabilities that are sharp in many cases of interest. We prove these distribution-free probabilistic inequalities by showing th…
In this paper, we establish an extension of a noncommutative Bennett inequality with a parameter $1\leq r\leq2$ and use it together with some noncommutative techniques to establish a Rosenthal inequality. We also present a noncommutative Hoeffding inequality as follows: Let $(\mathfrak{M}, τ)$ be a noncommutative probability space, $\mathfrak{N}$ be a von Neumann subalgebra of $\mathfrak{M}$ with the corresponding conditional expectation $\mathcal{E}_{\mathfrak{N}}$ and let subalgebras $\mathfrak{N}\subseteq\mathfrak{A}_j\subseteq\mathfrak{M}\,\,(j=1, \cdots, n)$ be successively independent ov…
Department of Mathematics, University of Rochester（罗切斯特大学数学系）；Institute of Mathematics at NAS RA（亚美尼亚国家科学院数学研究所）；Yerevan State University（亚美尼亚国立耶莱van大学）；Department of Mathematics, Binghamton University（宾夕法尼亚大学伯灵顿分校数学系）
The concentration of measure phenomenon may be summarized as follows: a function of many weakly dependent random variables that is not too sensitive to any of its individual arguments will tend to take values very close to its expectation. This phenomenon is most completely understood when the arguments are mutually independent random variables, and there exist several powerful complementary methods for proving concentration inequalities, such as the martingale method, the entropy method, and the method of transportation inequalities. The setting of dependent arguments is much less well unders…
