Starting from concentration of measure hypotheses on $m$ random vectors $Z_1,\ldots, Z_m$, this article provides an expression of the concentration of functionals $ϕ(Z_1,\ldots, Z_m)$ where the variations of $ϕ$ on each variable depend on the product of the norms (or semi-norms) of the other variables (as if $ϕ$ were a product). We illustrate the importance of this result through various generalizations of the Hanson-Wright concentration inequality as well as through a study of the random matrix $XDX^T$ and its resolvent $Q = (I_p - \frac{1}{n}XDX^T)^{-1}$, where $X$ and $D$ are random, which have fundamental interest in statistical machine learning applications.
In this expository note, we give a modern proof of Hanson-Wright inequality for quadratic forms in sub-gaussian random variables. We deduce a useful concentration inequality for sub-gaussian random vectors. Two examples are given to illustrate these results: a concentration of distances between random vectors and subspaces, and a bound on the norms of products of random and deterministic matrices.
We prove that quadratic forms in isotropic random vectors $X$ in $\mathbb{R}^n$, possessing the convex concentration property with constant $K$, satisfy the Hanson-Wright inequality with constant $CK$, where $C$ is an absolute constant, thus eliminating the logarithmic (in the dimension) factors in a recent estimate by Vu and Wang. We also show that the concentration inequality for all Lipschitz functions implies a uniform version of the Hanson-Wright inequality for suprema of quadratic forms (in the spirit of the inequalities by Borell, Arcones-Giné and Ledoux-Talagrand). Previous results of …
The Hanson-Wright inequality is an upper bound for tails of real quadratic forms in independent subgaussian random variables. In this work, we extend the Hanson-Wright inequality for the maximum eigenvalue of the quadratic sum of random Hermitian tensors under Einstein product. We first prove Weyl inequality for tensors under Einstein product and apply this fact to separate the quadratic form of random Hermitian tensors into diagonal sum and coupling (non-diagonal) sum parts. For the diagonal part, we can apply Bernstein inequality to bound the tail probability of the maximum eigenvalue of the…
The Hanson-Wright inequality is an upper bound for tails of real quadratic forms in independent random variables. In this work, we extend the Hanson-Wright inequality for the Ky Fan k-norm for the polynomial function of the quadratic sum of random tensors under Einstein product. We decompose the quadratic tensors sum into the diagonal part and the coupling part. For the diagonal part, we can apply the generalized tensor Chernoff bound directly. But, for the coupling part, we have to apply decoupling method first, i.e., decoupling inequality to bound expressions with dependent random tensors wi…
We derive a dimension-free Hanson-Wright inequality for quadratic forms of independent sub-gaussian random variables in a separable Hilbert space. Our inequality is an infinite-dimensional generalization of the classical Hanson-Wright inequality for finite-dimensional Euclidean random vectors. We illustrate an application to the generalized $K$-means clustering problem for non-Euclidean data. Specifically, we establish the exponential rate of convergence for a semidefinite relaxation of the generalized $K$-means, which together with a simple rounding algorithm imply the exact recovery of the t…
We provide moment bounds for expressions of the type $(X^{(1)} \otimes \dots \otimes X^{(d)})^T A (X^{(1)} \otimes \dots \otimes X^{(d)})$ where $\otimes$ denotes the Kronecker product and $X^{(1)}, \dots, X^{(d)}$ are random vectors with independent, mean 0, variance 1, subgaussian entries. The bounds are tight up to constants depending on $d$ for the case of Gaussian random vectors. Our proof also provides a decoupling inequality for expressions of this type. Using these bounds, we obtain new, improved concentration inequalities for expressions of the form $\|B (X^{(1)} \otimes \dots \otimes…
We discuss two-sided bounds for moments and tails of quadratic forms in Gaussian random variables with values in Banach spaces. We state a natural conjecture and show that it holds up to additional logarithmic factors. Moreover in a certain class of Banach spaces (including $L_r$-spaces) these logarithmic factors may be eliminated. As a corollary we derive upper bounds for tails and moments of quadratic forms in subgaussian random variables, which extend the Hanson-Wright inequality.
This paper is devoted to uniform versions of the Hanson-Wright inequality for a random vector $X \in \mathbb{R}^n$ with independent subgaussian components. The core technique of the paper is based on the entropy method combined with truncations of both gradients of functions of interest and of the coordinates of $X$ itself. Our results recover, in particular, the classic uniform bound of Talagrand (1996) for Rademacher chaoses and the more recent uniform result of Adamczak (2015), which holds under certain rather strong assumptions on the distribution of $X$. We provide several applications of…
We prove extensions of classical concentration inequalities for random variables which have $α$-subexponential tail decay for any $α\in (0,2]$. This includes Hanson--Wright type and convex concentration inequalities. We also provide some applications of these results. This includes uniform Hanson--Wright inequalities and concentration results for simple random tensors in the spirit of previous work by Klochkov--Zhivotovskiy and Vershynin.
In this paper, we provide a proof for the Hanson-Wright inequalities for sparsified quadratic forms in subgaussian random variables. This provides useful concentration inequalities for sparse subgaussian random vectors in two ways. Let $X = (X_1, \ldots, X_m) \in \mathbb{R}^m$ be a random vector with independent subgaussian components, and $ξ=(ξ_1, \ldots, ξ_m) \in \{0, 1\}^m$ be independent Bernoulli random variables. We prove the large deviation bound for a sparse quadratic form of $(X \circ ξ)^T A (X \circ ξ)$, where $A \in \mathbb{R}^{m \times m}$ is an $m \times m$ matrix, and random vect…
We present precise multilevel exponential concentration inequalities for polynomials in Ising models satisfying the Dobrushin condition. The estimates have the same form as two-sided tail estimates for polynomials in Gaussian variables due to Latała. In particular, for quadratic forms we obtain a Hanson-Wright type inequality. We also prove concentration results for convex functions and estimates for nonnegative definite quadratic forms, analogous as for quadratic forms in i.i.d. Rademacher variables, for more general random vectors satisfying the approximate tensorization property for entropy…
We consider a generic modified logarithmic Sobolev inequality (mLSI) of the form $\mathrm{Ent}_μ(e^f) \le \tfracρ{2} \mathbb{E}_μe^f Γ(f)^2$ for some difference operator $Γ$, and show how it implies two-level concentration inequalities akin to the Hanson--Wright or Bernstein inequality. This can be applied to the continuous (e.\,g. the sphere or bounded perturbations of product measures) as well as discrete setting (the symmetric group, finite measures satisfying an approximate tensorization property, \ldots). Moreover, we use modified logarithmic Sobolev inequalities on the symmetric group $S…
In this work we derive multi-level concentration inequalities for polynomial functions in independent random variables with a $α$-sub-exponential tail decay. A particularly interesting case is given by quadratic forms $f(X_1, \ldots, X_n) = \langle X,A X \rangle$, for which we prove Hanson-Wright-type inequalities with explicit dependence on various norms of the matrix $A$. A consequence of these inequalities is a two-level concentration inequality for quadratic forms in $α$-sub-exponential random variables, such as quadratic Poisson chaos. We provide various applications of these inequalities…
In this paper I prove good estimates on the moments and tail distribution of $k$-fold Wiener--Itô integrals and also present their natural counterpart for polynomials of independent Gaussian random variables. The proof is based on the so-called diagram formula for Wiener--Itô integrals which yields a good representation for their products as a sum of such integrals. I intend to show in a subsequent paper that this method also yields good estimates for degenerate $U$-statistics. The main result of this paper is a generalization of the estimates of Hanson and Wright about bilinear forms of indep…
