We give a concentration inequality based on the premise that random variables take values within a particular region. The concentration inequality guarantees that, for any sequence of correlated random variables, the difference between the sum of conditional expectations and that of the observed values takes a small value with high probability when the expected values are evaluated under the condition that the past values are known. Our inequality outperforms other well-known inequalities, e.g. the Azuma-Hoeffding inequality, especially in terms of the convergence speed when the random variables are highly biased. This high performance of our inequality is provided by the key idea in which we predict some parameters and adopt the predicted values in the inequality.
In this note, we obtain a Gaussian concentration inequality for a class of non-Lipschitz functions. In the one-dimensional case, our results supplement those established by Paouris and Valettas in [8].
In this expository note, we give a modern proof of Hanson-Wright inequality for quadratic forms in sub-gaussian random variables. We deduce a useful concentration inequality for sub-gaussian random vectors. Two examples are given to illustrate these results: a concentration of distances between random vectors and subspaces, and a bound on the norms of products of random and deterministic matrices.
We obtain a concentration inequality for the maximum degree of a vertex in a uniformly random dissection of a polygon. This resolves a conjecture posed by Curien and Kortchemski in 2012. Our approach is based on a bijection with dual trees and the tools of analytic combinatorics.
We prove that a probability measure on an abstract metric space satisfies a non trivial dimension free concentration inequality for the $\ell_2$ metric if and only if it satisfies the Poincaré inequality.
We present a proof of the concentration inequality for a discrete random surface model, where the underlying potential is perturbed by an additive random potential. The proof is based on annealing the random potential, and follows the method of [CEP96] and other works. Our result demonstrates the robustness of this method.
In this paper we revisit Talagrand's proof of concentration inequality for empirical processes. We give a different shorter proof of the main technical lemma that guarantees the existence of a certain kernel. Our proof provides the almost optimal value of the constant involved in the statement of this lemma.
In this paper a concentration inequality is proved for the deviation in the ergodic theorem in the case of discrete time observations of diffusion processes. The proof is based on the geometric ergodicity property for diffusion processes. As an application we consider the nonparametric pointwise estimation problem for the drift coefficient under discrete time observations.
The paper reexamines an argument by Talagrand that leads to a remarkable exponential tail bound for the concentration of probability near a set. The main novelty is the replacement of a mysterious Calculus inequality by an application of Jensen's inequality.
We prove an elementary yet useful inequality bounding the maximal value of certain linear programs. This leads directly to a bound on the martingale difference for arbitrarily dependent random variables, providing a generalization of some recent concentration of measure results. The linear programming inequality may be of independent interest.
The classical Gaussian concentration inequality for Lipschitz functions is adapted to a setting where the classical assumptions (i.e. Lipschitz and Gaussian) are not met. The theory is more direct than much of the existing theory designed to handle related generalizations. An application is presented to linear combinations of heavy tailed random variables.
We prove a concentration inequality which asserts that, under some mild regularity conditions, every random variable defined on the product of sufficiently many probability spaces exhibits pseudorandom behavior.
We estimate the concentration functions of $n$-fold convolutions of one-dimensional probability measures. The main result is a supplement to the results of Götze and Zaitsev (1998). We show that the estimation of concentration functions at arguments of bounded size can be reduced to the estimation of these functions at arguments of size $O(\sqrt n)$ which is easier.
We obtain a Bernstein type Gaussian concentration inequality for martingales. Our inequality improves the Azuma-Hoeffding inequality for moderate deviations $x$. Following the work of McDiarmid (1989), Talagrand (1996) and Boucheron, Lugosi and Massart (2000,2003), we show that our result can be applied to the concentration of random functions, Erdös-Rényi random graph, and maxima of empirical processes. Several interesting Gaussian concentration inequalities have been obtained.
We derive concentration inequalities for empirical means $\frac{1}{t} \int_0^t f(X_s) ds$ where $X_s$ is an irreducible Markov jump process on a finite state space and $f$ some observable. Using a Feynman-Kac semigroup we first derive a general concentration inequality. Then, based on this inequality we derive further concentration inequalities. Hereby we use three different methods; perturbation theory, Poincaré inequalities and information inequalities. We also obtain a Bernstein type concentration inequality.
We provide a sharp quantitative version of the Gaussian concentration inequality: for every $r>0$, the difference between the measure of the $r$-enlargement of a given set and the $r$-enlargement of a half-space controls the square of the measure of the symmetric difference between the set and a suitable half-space. We also prove a similar estimate in the Euclidean setting for the enlargement with a general convex set. This is equivalent to the stability of the Brunn-Minkowski inequality for the Minkowski sum between a convex set and a generic one.
The concentration inequality approach for normal approximation by Stein's method is generalized to the multivariate setting. We use this approach to prove a non-smooth function distance for multivariate normal approximation for standardized sums of $k$-dimensional independent random vectors $W=\sum_{i=1}^n X_i$ with an error bound of order $k^{1/2}γ$ where $γ=\sum_{i=1}^n E|X_i|^3$. For sums of locally dependent (unbounded) random vectors, we obtain a fourth moment bound which is typically of order $O_k(1/\sqrt{n})$, as well as a third moment bound which is typically of order $O_k(\log n/\sqrt…
We prove that quadratic forms in isotropic random vectors $X$ in $\mathbb{R}^n$, possessing the convex concentration property with constant $K$, satisfy the Hanson-Wright inequality with constant $CK$, where $C$ is an absolute constant, thus eliminating the logarithmic (in the dimension) factors in a recent estimate by Vu and Wang. We also show that the concentration inequality for all Lipschitz functions implies a uniform version of the Hanson-Wright inequality for suprema of quadratic forms (in the spirit of the inequalities by Borell, Arcones-Giné and Ledoux-Talagrand). Previous results of …
An inequality of K. Marton shows that the joint distribution of a Markov chain with uniformly contracting transition kernels exhibits concentration. We prove an analogous inequality for broadcast models on finite trees. We use this inequality to develop a condition for the sequence of depth-$k$ marginals of a broadcast model on a rooted infinite tree to form a normal Lévy family in terms of the Lipschitz constants of the transition kernels and the growth rate of the tree.
We prove a sharp inequality conjectured by Bobkov on the measure of dilations of Borel sets in $\mathbb{R}^n$ by a $s$-concave probability. Our result gives a common generalization of an inequality of Nazarov, Sodin and Volberg and a concentration inequality of Guédon. Applying our inequality to the level sets of functions satisfying a Remez type inequality, we deduce, as it is classical, that these functions enjoy dimension free distribution inequalities and Kahane-Khintchine type inequalities with positive and negative exponent, with respect to an arbitrary $s$-concave probability.
Recent development in high-dimensional statistical inference has necessitated concentration inequalities for a broader range of random variables. We focus on sub-Weibull random variables, which extend sub-Gaussian or sub-exponential random variables to allow heavy-tailed distributions. This paper presents concentration inequalities for independent sub-Weibull random variables with finite Generalized Bernstein-Orlicz norms, providing generalized Bernstein's inequalities and Rosenthal-type moment bounds. The tightness of the proposed bounds is shown through lower bounds of the concentration ineq…
We present a non-asymptotic concentration inequality for the random matrix product \begin{equation}\label{eq:Zn} Z_n = \left(I_d-αX_n\right)\left(I_d-αX_{n-1}\right)\cdots \left(I_d-αX_1\right), \end{equation} where $\left\{X_k \right\}_{k=1}^{+\infty}$ is a sequence of bounded independent random positive semidefinite matrices with common expectation $\mathbb{E}\left[X_k\right]=Σ$. Under these assumptions, we show that, for small enough positive $α$, $Z_n$ satisfies the concentration inequality \begin{equation}\label{eq:CTbound} \mathbb{P}\left(\left\Vert Z_n-\mathbb{E}\left[Z_n\right]\right\V…
Given two vectors in Euclidean space, how unlikely is it that a random vector has a larger inner product with the shorter vector than with the longer one? When the random vector has independent, identically distributed components, we conjecture that this probability is no more than a constant multiple of the ratio of the Euclidean norms of the two given vectors, up to an additive term to allow for the possibility that the longer vector has more arithmetic structure. We give some partial results to support the basic conjecture.
We prove a new concentration inequality for U-statistics of order two for uniformly ergodic Markov chains. Working with bounded and $π$-canonical kernels, we show that we can recover the convergence rate of Arcones and Gin{é} who proved a concentration result for U-statistics of independent random variables and canonical kernels. Our result allows for a dependence of the kernels $h_{i,j}$ with the indexes in the sums, which prevents the use of standard blocking tools. Our proof relies on an inductive analysis where we use martingale techniques, uniform ergodicity, Nummelin splitting and Bernst…
We give a concentration inequality for a stochastic version of the facility location problem. We show the objective $C_n = \min_{F \subseteq [0,1]^2}|F|+\sum_{x\in X}\min_{f\in F}\|x-f\|$ is concentrated in an interval of length $O(n^{1/6})$ and $\E[C_n]=Θ(n^{2/3})$ if the input $X$ consists of i.i.d. uniform points in the unit square. Our main tool is to use a geometric quantity, previously used in the design of approximation algorithms for the facility location problem, to analyze a martingale process. Many of our techniques generalize to other settings.
For a map of the unit interval with an indifferent fixed point, we prove an upper bound for the variance of all observables of $n$ variables $K:[0,1]^n\to\R$ which are componentwise Lipschitz. The proof is based on coupling and decay of correlation properties of the map. We then give various applications of this inequality to the almost-sure central limit theorem, the kernel density estimation, the empirical measure and the periodogram.
In this paper we study the regularity of paths in terms of properties of admissible nets. We show the right concentration inequality above the modulus of continuity. Using the approach we prove the Bernstein type inequality for the empirical processes. Therefore we obtain the best form of concentration for processes studied recently by Mendelson and Paouris. Results of this type are of importance in the compressed sensing theory.
Initially motivated by the study of the non-asymptotic properties of non-parametric tests based on permutation methods, concentration inequalities for uniformly permuted sums have been largely studied in the literature. Recently, Delyon et al. proved a new Bernstein-type concentration inequality based on martingale theory. This work presents a new proof of this inequality based on the fundamental inequalities for random permutations of Talagrand. The idea is to first obtain a rough inequality for the square root of the permuted sum, and then, iterate the previous analysis and plug this first i…
We introduce a symmetrization technique that allows us to translate a problem of controlling the deviation of some functionals on a product space from their mean into a problem of controlling the deviation between two independent copies of the functional. As an application we give a new easy proof of Talagrand's concentration inequality for empirical processes, where besides symmetrization we use only Talagrand's concentration inequality on the discrete cube {-1,+1}^n. As another application of this technique we prove new Vapnik-Chervonenkis type inequalities. For example, for VC-classes of fu…
We provide a simple, general argument to obtain improvements of concentration-type inequalities starting from improvements of their corresponding isoperimetric-type inequalities. We apply this argument to obtain robust improvements of the Brunn-Minkowski inequality (for Minkowski sums between generic sets and convex sets) and of the Gaussian concentration inequality. The former inequality is then used to obtain a robust improvement of the Riesz rearrangement inequality under certain natural conditions. These conditions are compatible with the applications to a finite-range nonlocal isoperimetr…
Let X_1 ,..., X_n be a collection of binary valued random variables and let f : {0,1}^n -> R be a Lipschitz function. Under a negative dependence hypothesis known as the {\em strong Rayleigh} condition, we show that f - E f satisfies a concentration inequality generalizing the classical Gaussian concentration inequality for sums of independent Bernoullis: P (S_n - E S_n > a) < exp (-2 a^2 / n). The class of strong Rayleigh measures includes determinantal measures, weighted uniform matroids and exclusion measures; some familiar examples from these classes are generalized negative binomials and …
We prove concentration inequalities for $f\left( X\right) $ about its median, where $X$ is a random vector in $\mathbb{R}^n$ with independent heavy tailed coordinates of Weibull or power type, and $f:\mathbb{R}^n\rightarrow\mathbb{R}$ is a locally Lipschitz function. This paper is part of a series of four papers, Part I, Part II and two supporting papers. It can be read independently of Part I.
This paper deduces exponential matrix concentration from a Poincaré inequality via a short, conceptual argument. Among other examples, this theory applies to matrix-valued functions of a uniformly log-concave random vector. The proof relies on the subadditivity of Poincaré inequalities and a chain rule inequality for the trace of the matrix Dirichlet form. It also uses a symmetrization technique to avoid difficulties associated with a direct extension of the classic scalar argument.
We prove an exponential decay concentration inequality to bound the tail probability of the difference between the log-likelihood of discrete random variables on a finite alphabet and the negative entropy. The concentration bound we derive holds uniformly over all parameter values. The new result improves the convergence rate in an earlier result of Zhao (2020), from $(K^2\log K)/n=o(1)$ to $ (\log K)^2/n=o(1)$, where $n$ is the sample size and $K$ is the size of the alphabet. We further prove that the rate $(\log K)^2/n=o(1)$ is optimal. The results are extended to misspecified log-likelihood…
We present concentration inequalities on the multislice which are based on (modified) log-Sobolev inequalities. This includes bounds for convex functions and multilinear polynomials. As an application we show concentration results for the triangle count in the $G(n,M)$ Erdős--Rényi model resembling known bounds in the $G(n,p)$ case. Moreover, we give a proof of Talagrand's convex distance inequality for the multislice. Interpreting the multislice in a sampling without replacement context, we furthermore present concentration results for $n$ out of $N$ sampling without replacement. Based on a b…
We present an adaptation of the Kato--Temple inequality for bounding perturbations of eigenvalues with applications to statistical inference for random graphs, specifically hypothesis testing and change-point detection. We obtain explicit high-probability bounds for the individual distances between certain signal eigenvalues of a graph's adjacency matrix and the corresponding eigenvalues of the model's edge probability matrix, even when the latter eigenvalues have multiplicity. Our results extend more broadly to the perturbation of singular values in the presence of quite general random matrix…
We present novel martingale concentration inequalities for martingale differences with finite Orlicz-$ψ_α$ norms. Such martingale differences with weak exponential-type tails scatters in many statistical applications and can be heavier than sub-exponential distributions. In the case of one dimension, we prove in general that for a sequence of scalar-valued supermartingale difference, the tail bound depends solely on the sum of squared Orlicz-$ψ_α$ norms instead of the maximal Orlicz-$ψ_α$ norm, generalizing the results of Lesigne & Volný (2001) and Fan et al. (2012). In the multidimensional ca…
We consider a generic modified logarithmic Sobolev inequality (mLSI) of the form $\mathrm{Ent}_μ(e^f) \le \tfracρ{2} \mathbb{E}_μe^f Γ(f)^2$ for some difference operator $Γ$, and show how it implies two-level concentration inequalities akin to the Hanson--Wright or Bernstein inequality. This can be applied to the continuous (e.\,g. the sphere or bounded perturbations of product measures) as well as discrete setting (the symmetric group, finite measures satisfying an approximate tensorization property, \ldots). Moreover, we use modified logarithmic Sobolev inequalities on the symmetric group $S…
In this work we design a general method for proving moment inequalities for polynomials of independent random variables. Our method works for a wide range of random variables including Gaussian, Boolean, exponential, Poisson and many others. We apply our method to derive general concentration inequalities for polynomials of independent random variables. We show that our method implies concentration inequalities for some previously open problems, e.g. permanent of a random symmetric matrices. We show that our concentration inequality is stronger than the well-known concentration inequality due …
We show that any probability measure satisfying a Matrix Poincaré inequality with respect to some reversible Markov generator satisfies an exponential matrix concentration inequality depending on the associated matrix carré du champ operator. This extends to the matrix setting a classical phenomenon in the scalar case. Moreover, the proof gives rise to new matrix trace inequalities which could be of independent interest. We then apply this general fact by establishing matrix Poincaré inequalities to derive matrix concentration inequalities for Gaussian measures, product measures and for Strong…
Let H=A+UBU* where A and B are two N-by-N Hermitian matrices and U is a Haar-distributed random unitary matrix, and let μ_H, μ_A, and μ_B be empirical measures of eigenvalues of matrices H, A, and B, respectively. Then, it is known (see, for example, Pastur-Vasilchuk, CMP, 2000, v.214, pp.249-286) that for large N, measure μ_H is close to the free convolution of measures μ_A and μ_B, where the free convolution is a non-linear operation on probability measures. The large deviations of the cumulative distribution function of μ_H from its expectation have been studied by Chatterjee in in JFA, 200…
Random union sets $Z$ associated with stationary Poisson processes of $k$-cylinders in $\mathbb{R}^d$ are considered. Under general conditions on the typical cylinder base a concentration inequality for the volume of $Z$ restricted to a compact window is derived. Assuming convexity of the typical cylinder base and isotropy of $Z$ a concentration inequality for intrinsic volumes of arbitrary order is established. A number of special cases are discussed, for example the case when the cylinder bases arise from a random rotation of a fixed convex body. Also the situation of expanding windows is st…
In the paper we study the infimum convolution inequalites. Such an inequality was first introduced by B. Maurey to give the optimal concentration of measure behaviour for the product exponential measure. We show how IC-inequalities are tied to concentration and study the optimal cost functions for an arbitrary probability measure. In particular, we show the optimal IC-inequality for product log-concave measures and for uniform measures on the l_p^n balls. Such an optimal inequality implies, for a given measure, in particular the Central Limit Theorem of Klartag and the tail estimates of Paouri…
If a random variable is not exponentially integrable, it is known that no concentration inequality holds for an infinite sequence of independent copies. Under mild conditions, we establish concentration inequalities for finite sequences of $n$ independent copies, with good dependence in $n$.
Let $Y_v, v\in V,$ be $[0,1]$-valued random variables having a dependency graph $G=(V,E)$. We show that \[ \mathbb{E}\left[\prod_{v\in V} Y_{v} \right] \leq \prod_{v\in V} \left\{ \mathbb{E}\left[Y_v^{\frac{χ_b}{b}}\right] \right\}^{\frac{b}{χ_b}}, \] where $χ_b$ is the $b$-fold chromatic number of $G$. This inequality may be seen as a dependency-graph analogue of a generalised Hölder inequality, due to Helmut Finner. Additionally, we provide applications of Hölder-type inequalities to concentration and correlation bounds for sums of weakly dependent random variables.
We show that the number of real roots of random trigonometric polynomials with i.i.d. coefficients, which are either bounded or satisfy the logarithmic Sobolev inequality, satisfies an exponential concentration of measure.
Inequalities between transportation costs and Fisher information are known to characterize certain concentration properties of Markov processes around their invariant measures. This note provides a new characterization of the quadratic transportation-information inequality $W_2I$ in terms of a dimension-free concentration property for i.i.d. (conditionally on the initial positions) copies of the underlying Markov process. This parallels Gozlan's characterization of the quadratic transportation-entropy inequality $W_2H$. The proof is based on a new Laplace-type principle for the operator norms …
In this paper, we consider Poincaré inequalities for non euclidean metrics on $\mathbb{R}^d$. These inequalities enable us to derive precise dimension free concentration inequalities for product measures. This technique is appropriate for a large scope of concentration rate: between exponential and gaussian and beyond. We give different equivalent functional forms of these Poincaré type inequalities in terms of transportation-cost inequalities and infimum convolution inequalities. Workable sufficient conditions are given and a comparison is made with generalized Beckner-Latala-Oleszkiewicz ine…
The aim of this paper is to show that a probability measure concentrates independently of the dimension like a gaussian measure if and only if it verifies Talagrand's $\T_2$ transportation-cost inequality. This theorem permits us to give a new and very short proof of a result of Otto and Villani. Generalizations to other types of concentration are also considered. In particular, one shows that the Poincaré inequality is equivalent to a certain form of dimension free exponential concentration. The proofs of these results rely on simple Large Deviations techniques.
The Entropy method provides a powerful framework for proving scalar concentration inequalities by establishing functional inequalities like Poincare and log-Sobolev inequalities. These inequalities are especially useful for deriving concentration for dependent distributions coming from stationary distributions of Markov chains. In contrast to the scalar case, the study of matrix concentration inequalities is fairly recent and there does not seem to be a general framework for proving matrix concentration inequalities using the Entropy method. In this paper, we initiate the study of Matrix value…
We explore the question whether Lipschitz functions of random variables under various forms of negative correlation satisfy concentration bounds similar to McDiarmid's inequality for independent random variables. We prove such a concentration bound for random variables satisfying the condition of negative regression, correcting an earlier proof by Dubhashi and Ranjan.
