A survey is given of some Chernoff type bounds for the tail probabilities P(X-EX > a) and P(X-EX < a) when X is a random variable that can be written as a sum of indicator variables that are either independent or negatively related. Most bounds are previously known and some comparisons are made.
This paper was written in 1994, but was never published because I had overlooked some existing papers containing some of the inequalities. Because of some recent interest in one of the inequalities, which does not seem to be published anywhere else, it has now been lightly edited and made available here.
We give exponential upper bounds for $P(S \le k)$, in particular $P(S=0)$, where $S$ is a sum of indicator random variables that are positively associated. These bounds allow, in particular, a comparison with the independent case. We give examples in which we compare with a famous exponential inequality for sums of correlated indicators, the Janson inequality. Here our bound sometimes proves to be superior to Janson's bound.
We study the exact constants in the moment inequalities for sums of centered independent random variables: improve their asymptotics, low and upper bounds, calculate more exact asymptotics, elaborate the numerical algorithm for their calculation, study the class of smoothing etc.
Under Poincaré-type conditions, upper bounds are explored for the Kolmogorov distance between the distributions of weighted sums of dependent summands and the normal law. Based on improved concentration inequalities on high-dimensional Euclidean spheres, the results extend and refine previous results to non-symmetric models.
In this paper we give new deviation inequalities of Bernstein's type for the partial sums of weakly dependent time series. The loss from the independent case is studied carefully. We give non mixing examples such that dynamical systems and Bernoulli shifts for whom our deviation inequalities hold. The proofs are based on the blocks technique and different coupling arguments.
Random matrices have played an important role in many fields including machine learning, quantum information theory and optimization. One of the main research focuses is on the deviation inequalities for eigenvalues of random matrices. Although there are intensive studies on the large-deviation inequalities for random matrices, only a few of works discuss the small-deviation behavior of random matrices. In this paper, we present the small-deviation inequalities for the largest eigenvalues of sums of random matrices. Since the resulting inequalities are independent of the matrix dimension, they…
Let $X_1,X_2,\ldots,X_n$ be independent random variables and $S_k=\sum_{i=1}^k X_i$. We show that for any constants $a_k$, \[ \Pr(\max_{1\leq k\leq n}||S_{k}|-a_{k}|>11t)\leq 30 \max_{1\leq k\leq n}\Pr(||S_{k}|-a_{k}|>t). \] We also discuss similar inequalities for sums of Hilbert and Banach space valued random vectors.
We establish deviation inequalities for the maxima of partial sums of a martingale differences sequence, and of a strictly stationary orthomartingale random field. These inequalities can be used to establish complete convergence of normalized maxima of partial sums.
The aim of this paper is to propose new Rosenthal-type inequalities for moments of order higher than 2 of the maximum of partial sums of stationary sequences including martingales and their generalizations. As in the recent results by Peligrad et al. [Proc. Amer. Math. Soc. 135 (2007) 541-550] and Rio [J. Theoret. Probab. 22 (2009) 146-163], the estimates of the moments are expressed in terms of the norms of projections of partial sums. The proofs of the results are essentially based on a new maximal inequality generalizing the Doob maximal inequality for martingales and dyadic induction. Vari…
Let $\{X_i,i\geq1\}$ be a sequence of negatively associated random variables, and let $\{X_i^\ast,i\geq 1\}$ be a sequence of independent random variables such that $X_i^\ast$ and $X_i$ have the same distribution for each $i$. Denote by $S_k=\sum_{i=1}^{k}X_i$ and $S_k^\ast=\sum_{i=1}^{k}X_i^\ast$ for $k\geq 1$. The well-known results of Shao \cite{Shao2000} sates that $\mathbb{E}f(S_n)\leq \mathbb{E}f(S_n^\ast)$ for any nondecreasing convex function. Using this very strong property, we obtain a large variety of deviation inequalities for $S_n$
In this work we present concentration inequalities for the sum $S_n$ of independent integer-valued not necessary indentically distributed random variables, where each variable has tail function that can be bounded by some power function with exponent $-α$. We show that when $0<α\leq 1$, then the sum does not have finite expectation, however, with high probability we have that $|S_n|=O\left(n^{1/α}\right)$. When $α>1$, then the sum $S_n$ is concentrated around its mean. Since the r.vs. that constitute the sum has tails, which can be bounded by some power function, it follows that results of thi…
The long-standing Gaussian product inequality (GPI) conjecture states that $E [\prod_{j=1}^{n}X_j^{2m_j}]\geq\prod_{j=1}^{n}E[X_j^{2m_j}]$ for any centered Gaussian random vector $(X_1,\dots,X_n)$ and $m_1,\dots,m_n\in\mathbb{N}$. In this paper, we describe a computational algorithm involving sums-of-squares representations of multivariate polynomials that can be used to resolve the GPI conjecture. To exhibit the power of this novel method, we apply it to prove two new GPIs: $E[X_1^{2m_1}X_2^{6}X_3^{4}]\ge E[X_1^{2m_1}]E[X_2^{6}]E[X_3^{4}]$ and $E[X_1^{2m_1}X_2^{2}X_3^{2}X_4^{2}]\ge E[X_1^{2m_…
In this paper, we are interested in the limit theorem question for sums of indicator functions. We show that in every aperiodic dynamical system, for every increasing sequence $(a_n)_{n\in\N}\subset\R_+$ such that $a_n\nearrow\infty$ and $\frac{a_n}{n}\to 0$ as $n\to\infty$, there exists a measurable set $A$ such that the sequence of the distributions of the partial sums $\frac{1}{a_n}\sum_{i=0}^{n-1}(\ind_A-μ(A))\circ T^i$ is dense in the set of the probability measures on $\R$. Further, in the ergodic case, we prove that there exists a dense $G_δ$ of such sets.
Laws of the iterated and single logarithm for sums of independent indicators, with applications to the Ginibre point process and Karlin's occupancy scheme
Certain previously known upper bounds on the moments of the norm of martingales in 2-smooth Banach spaces are improved. Some of these improvements hold even for sums of independent real-valued random variables. Applications to concentration of measure on product spaces for separately Lipschitz functions are presented, including ones concerning the central moments of the norm of the sums of independent random vectors in any separable Banach space.
Rutgers University（罗格斯大学）；University of Pennsylvania（宾夕法尼亚大学）
We derive exponential tail inequalities for sums of random matrices with no dependence on the explicit matrix dimensions. These are similar to the matrix versions of the Chernoff bound and Bernstein inequality except with the explicit matrix dimensions replaced by a trace quantity that can be small even when the dimension is large or infinite. Some applications to principal component analysis and approximate matrix multiplication are given to illustrate the utility of the new bounds.
This paper presents new probability inequalities for sums of independent, random, self-adjoint matrices. These results place simple and easily verifiable hypotheses on the summands, and they deliver strong conclusions about the large-deviation behavior of the maximum eigenvalue of the sum. Tail bounds for the norm of a sum of random rectangular matrices follow as an immediate corollary. The proof techniques also yield some information about matrix-valued martingales. In other words, this paper provides noncommutative generalizations of the classical bounds associated with the names Azuma, Benn…
Initially motivated by the study of the non-asymptotic properties of non-parametric tests based on permutation methods, concentration inequalities for uniformly permuted sums have been largely studied in the literature. Recently, Delyon et al. proved a new Bernstein-type concentration inequality based on martingale theory. This work presents a new proof of this inequality based on the fundamental inequalities for random permutations of Talagrand. The idea is to first obtain a rough inequality for the square root of the permuted sum, and then, iterate the previous analysis and plug this first i…
Lower bounds for the Rényi entropies of sums of independent random variables taking values in cyclic groups of prime order under permutations are established. The main ingredients of our approach are extended rearrangement inequalities in prime cyclic groups building on Lev (2001), and notions of stochastic ordering. Several applications are developed, including to discrete entropy power inequalities, the Littlewood-Offord problem, and counting solutions of certain linear systems.
Small ball inequalities have been extensively studied in the setting of Gaussian processes and associated Banach or Hilbert spaces. In this paper, we focus on studying small ball probabilities for sums or differences of independent, identically distributed random elements taking values in very general sets. Depending on the setting--abelian or nonabelian groups, or vector spaces, or Banach spaces--we provide a collection of inequalities relating different small ball probabilities that are sharp in many cases of interest. We prove these distribution-free probabilistic inequalities by showing th…
We calculate the exact subgaussian norm of a centered (shifted) indicator (Bernoulli's) random variable. Using this result we derive very simple tail estimates for sums of these variables, not necessary to be identical distributed, and give some examples to show the exactness of our estimates.
The exponential inequality of the maximum partial sums is a key to establish the law of the iterated logarithm of negatively associated random variables. In the one-indexed random sequence case, such inequalities for negatively associated random variables are established by Shao (2000) by using his comparison theorem between negatively associated and independent random variables. In the multi-indexed random field case, the comparison theorem fails. The purpose of this paper is to establish the Kolmogorov exponential inequality as well a moment inequality of the maximum partial sums of a negati…
Classical Kolmogorov's and Rosenthal's inequalities for the maximum partial sums of random variables are basic tools for studying the strong laws of large numbers. In this paper, motived by the notion of independent and identically distributed random variables under the sub-linear expectation initiated by Peng (2006, 2008b), we introduce the concept of negative dependence of random variables and establish Kolmogorov's and Rosenthal's inequalities for the maximum partial sums of negatively dependent random variables under the sub-linear expectations. As an application, we show that Kolmogorov's…
Under correlation-type conditions, we derive an upper bound of order $(\log n)/n$ for the average Kolmogorov distance between the distributions of weighted sums of dependent summands and the normal law. The result is based on improved concentration inequalities on high-dimensional Euclidean spheres. Applications are illustrated on the example of log-concave probability measures.
We obtain some optimal inequalities on tail probabilities for sums of independent bounded random variables. Our main result completes an upper bound on tail probabilities due to Talagrand by giving a one-term asymptotic expansion for large deviations. This result can also be regarded as sharp large deviations of types of Cramér and Bahadur-Ranga Rao.
The paper is devoted to establishing some general exponential inequalities for supermartingales. The inequalities improve or generalize many exponential inequalities of Bennett, Freedman, de la Peña, Pinelis and van de Geer. Moreover, our concentration inequalities also improve some known inequalities for sums of independent random variables. Applications associated with linear regressions, autoregressive processes and branching processes are provided. In particular, an interesting application of {de la Peña's} inequality to self-normalized deviations is also provided.
Correcting for skewness can result in more accurate tail probability approximations in the central limit theorem for sums of independent random variables. In this paper, we extend the theory to sums of local statistics of independent random variables and apply the result to $k$-runs, U-statistics, and subgraph counts in the Erdös-Rényi random graph. To prove our main result, we develop exponential concentration inequalities and higher-order Cramér-type moderate deviations via Stein's method.
We obtain moderate deviations theorems and exponential (Bernstein type) concentration inequalities for "nonconventional" sums of the form $S_N=\sum_{n=1}^N (F(ξ_{q_1(n)},ξ_{q_2(n)},...,ξ_{q_\ell(n)})-\bar F)$.
Let $\{X_n;n\ge 1\}$ be a sequence of independent random variables on a probability space $(Ω, \mathcal{F}, P)$ and $S_n=\sum_{k=1}^n X_k$. It is well-known that the almost sure convergence, the convergence in probability and the convergence in distribution of $S_n$ are equivalent. In this paper, we prove similar results for the independent random variables under the sub-linear expectations, and give a group of sufficient and necessary conditions for these convergence. For proving the results, the Levy and Kolmogorov maximal inequalities for independent random variables under the sub-linear ex…
Questions are posed regarding the influence that the column sums of the transition probabilities of a stochastic matrix (with row sums all one) have on the stationary distribution, the mean first passage times and the Kemeny constant of the associated irreducible discrete time Markov chain. Some new relationships, including some inequalities, and partial answers to the questions, are given using a special generalized matrix inverse that has not previously been considered in the literature on Markov chains.
The approach of Kleitman (1970) and Kanter (1976) to multivariate concentration function inequalities is generalized in order to obtain for deviation probabilities of sums of independent symmetric random variables a lower bound depending only on deviation probabilities of the terms of the sum. This bound is optimal up to discretization effects, improves on a result of Nagaev (2001), and complements the comparison theorems of Birnbaum (1948) and Pruss (1997). Birnbaum's theorem for unimodal random variables is extended to the lattice case.
In this article, we provide an extension of the Chen-Stein inequality for Poisson approximation in the total variation distance for sums of independent Bernoulli random variables in two ways. We prove that we can improve the rate of convergence (hence the quality of the approximation) by using explicitly constructed signed or positive probability measures, and that we can extend the setting to possibly dependent random variables. The framework which allows this is that of mod-Poisson convergence, and more precisely those mod-Poisson convergent sequences whose residue functions can be expressed…
Let $X,X_1,\ldots,X_n$ be independent identically distributed random variables. In this paper we study the behavior of concentration functions of weighted sums $\sum_{k=1}^{n}a_k X_k$ with respect to the arithmetic structure of coefficients $a_k$ in the context of the Littlewood--Offord problem. Concentration results of this type received renewed interest in connection with distributions of singular values of random matrices. Recently, Tao and Vu proposed an Inverse Principle in the Littlewood-Offord problem. We discuss the relations between the Inverse Principle of Tao and Vu as well as that …
This work prepares new probability bounds for sums of random, independent, Hermitian tensors. These probability bounds characterize large-deviation behavior of the extreme eigenvalue of the sums of random tensors. We extend Lapalace transform method and Lieb's concavity theorem from matrices to tensors, and apply these tools to generalize the classical bounds associated with the names Chernoff, Bennett, and Bernstein from the scalar to the tensor setting. Tail bounds for the norm of a sum of random rectangular tensors are also derived from corollaries of random Hermitian tensors cases. The pro…
Let $η_{1},η_2,...$ be independent (not necessarily identically distributed) zero-mean random variables (r.v.'s) such that $|η_i|\le1$ almost surely for all $i$, and let $Z$ stand for a standard normal r.v. Let $a_1,a_2,...$ be any real numbers such that $a_1^2+a_2^2+...=1.$ It is shown that then $$ ¶(a_1η_1+a_2η_2+...\ge x) \le ¶(Z\ge x-\la/x) \forall x>0, $$ where $\la := \ln\frac{2e^3}9=1.495...$. The proof relies on (i) another probability inequality and (ii) a l'Hospital-type rule for monotonicity, both developed elsewhere. A multidimensional analogue of this result is given, based on a d…
We compare weighted sums of i.i.d. positive random variables according to the usual stochastic order. The main inequalities are derived using majorization techniques under certain log-concavity assumptions. Specifically, let $Y_i$ be i.i.d. random variables on $\mathbf{R}_+$. Assuming that $\log Y_i$ has a log-concave density, we show that $\sum a_iY_i$ is stochastically smaller than $\sum b_iY_i$, if $(\log a_1,...,\log a_n)$ is majorized by $(\log b_1,...,\log b_n)$. On the other hand, assuming that $Y_i^p$ has a log-concave density for some $p>1$, we show that $\sum a_iY_i$ is stochasticall…
Given a vector $F=(F_1,\dots,F_m)$ of Poisson functionals $F_1,\dots,F_m$, we investigate the proximity between $F$ and an $m$-dimensional centered Gaussian random vector $N_Σ$ with covariance matrix $Σ\in\mathbb{R}^{m\times m}$. Apart from finding proximity bounds for the $d_2$- and $d_3$-distances, based on classes of smooth test functions, we obtain proximity bounds for the $d_{convex}$-distance, based on the less tractable test functions comprised of indicators of convex sets. The bounds for all three distances are shown to be of the same order, which is presumably optimal. The bounds are …
The sumset and inverse sumset theories of Freiman, Plünnecke and Ruzsa, give bounds connecting the cardinality of the sumset $A+B=\{a+b\;;\;a\in A,\,b\in B\}$ of two discrete sets $A,B$, to the cardinalities (or the finer structure) of the original sets $A,B$. For example, the sum-difference bound of Ruzsa states that, $|A+B|\,|A|\,|B|\leq|A-B|^3$, where the difference set $A-B= \{a-b\;;\;a\in A,\,b\in B\}$. Interpreting the differential entropy $h(X)$ of a continuous random variable $X$ as (the logarithm of) the size of the effective support of $X$, the main contribution of this paper is a se…
A novel multinomial theorem for commutative idempotents is shown to lead to new results about the moments, central moments, factorial moments, and their generating functions for any random variable $X = \sum_{i} Y_i $ expressible as a sum of Bernoulli indicator random variables $Y_i$. The resulting expressions are functions of the expectation of products of the Bernoulli indicator random variables. These results are used to derive novel expressions for the various moments in a large number of familiar examples and classic problems including: the binomial, hypergeometric, Poisson limit of the b…
We prove moment inequalities for a class of functionals of i.i.d. random fields. We then derive rates in the central limit theorem for weighted sums of such randoms fields via an approximation by $m$-dependent random fields.
In this paper, we establish an extension of a noncommutative Bennett inequality with a parameter $1\leq r\leq2$ and use it together with some noncommutative techniques to establish a Rosenthal inequality. We also present a noncommutative Hoeffding inequality as follows: Let $(\mathfrak{M}, τ)$ be a noncommutative probability space, $\mathfrak{N}$ be a von Neumann subalgebra of $\mathfrak{M}$ with the corresponding conditional expectation $\mathcal{E}_{\mathfrak{N}}$ and let subalgebras $\mathfrak{N}\subseteq\mathfrak{A}_j\subseteq\mathfrak{M}\,\,(j=1, \cdots, n)$ be successively independent ov…
Let $\BS_1,...,\BS_n$ be independent identically distributed random variables each having the standardized Bernoulli distribution with parameter $p\in(0,1)$. Let $m_*(p):=(1+p+2p^2)/(2\sqrt{p-p^2}+4p^2)$ if $0<p\le 1/2$ and $m_*(p):=1$ if $1/2\le p<1$. Let $m\ge m_*(p)$. Let $f$ be such a function that $f$ and $f''$ are nondecreasing and convex. Then it is proved that for all nonnegative numbers $c_1,...,c_n$ one has the inequality $$\E f(c_1\BS_1+...+c_n\BS_n)\le\E f(s^{(m)}\cdot(\BS_1+...+\BS_n)),$$ where $s^{(m)}:=(\frac1n \sum_{i=1}^n c_i^{2m})^\frac1{2m}$. The lower bound $m_*(p)$ on $m$ …
