We prove a multivariate version of Bernstein's inequality about the probability that degenerate $U$-statistics take a value larger than some number $u$. This is an improvement of former estimates for the same problem which yields an asymptotically sharp estimate for not too large numbers $u$. This paper also contains an analogous bound about the distribution of multiple Wiener-Ito integrals. Their comparison shows that our results are sharp. The proofs are based on good estimates about high moments of multiple random integrals. They are obtained by means of a diagram formula which enables us to express the product of multiple random integrals as the sum of such expressions.
We consider the stochastic integrals of multivariate point processes and study their concentration phenomena. In particular, we obtain a Bernstein type of concentration inequality through Doléans-Dade exponential formula and a uniform exponential inequality using a generic chaining argument. As applications, we obtain a upper bound for a sequence of discrete time martingales indexed by a class of functionals, and so derive the rate of convergence for nonparametric maximum likelihood estimators, which is an improvement of earlier work of van de Geer.
It is be shown that the sequence of Bernstein polynomials for a function of several variables converges to this function uniformly along with every partial derivative of any order, provided that the latter derivative is well defined and continuous. This may be interesting in some questions of stochastic calculus, in particular, in a methodical proof of multidimensional Ito's formula based on the product rule.
We modify the classical Bernstein's inequality for the sums of independent centered random variables (r.v.) in the terms of relative tails or moments. We built also some examples in order to show the exactness of offered results.
In this paper we obtain a Bernstein type inequality for the sum of self-adjoint centered and geometrically absolutely regular random matrices with bounded largest eigenvalue. This inequality can be viewed as an extension to the matrix setting of the Bernstein-type inequality obtained by Merlevède et al. (2009) in the context of real-valued bounded random variables that are geometrically absolutely regular. The proofs rely on decoupling the Laplace transform of a sum on a Cantor-like set of random matrices.
We give a distribution-dependent concentration inequality for functions of independent variables. The result extends Bernstein's inequality from sums to more general functions, whose variation in any argument does not depend too much on the other arguments. Applications sharpen existing bounds for U-statistics and the generalization error of regularized least squares.
In this paper we obtain a Bernstein type inequality for a class of weakly dependent and bounded random variables. The proofs lead to a moderate deviations principle for sums of bounded random variables with exponential decay of the strong mixing coeficients that complements the large deviation result obtained by Bryc and Dembo (1998) under superexponential mixing rates.
In this paper we present a tail inequality for the maximum of partial sums of a weakly dependent sequence of random variables that are not necessarily bounded. The class considered includes geometrically and subgeometrically strongly mixing sequences. The result is then used to derive asymptotic moderate deviations results. Applications include classes of Markov chains, functions of linear processes with absolutely regular innovations and ARCH models
Using the renewal approach we prove Bernstein-like inequalities for additive functionals of geometrically ergodic Markov chains, thus obtaining counterparts of inequalities for sums of independent random variables. The coefficient in the sub-Gaussian part of our estimate is the asymptotic variance of the additive functional, i.e. the variance of the limiting Gaussian variable in the Central Limit Theorem for Markov chains. This refines earlier results by R. Adamczak and W. Bednorz, which were obtained under the additional assumption of strong aperiodicity of the chain.
We consider the three dimensional array $\mathcal{A} = \{a_{i,j,k}\}_{1\le i,j,k \le n}$, with $a_{i,j,k} \in [0,1]$, and the two random statistics $T_{1}:= \sum_{i=1}^n \sum_{j=1}^n a_{i,j,σ(i)}$ and $T_{2}:= \sum_{i=1}^{n} a_{i,σ(i),π(i)}$, where $σ$ and $π$ are chosen independently from the set of permutations of $\{1,2,\ldots,n \}.$ These can be viewed as natural three dimensional generalizations of the statistic $T_{3}=\sum_{i=1}^{n} a_{i,σ(i)}$, considered by Hoeffding \cite{Hoe51}. Here we give Bernstein type concentration inequalities for $T_{1}$ and $T_{2}$ by extending the argument f…
We show that the Bernstein-Hoeffding method can be employed to a larger class of generalized moments. This class includes the exponential moments whose properties play a key role in the proof of a well-known inequality of Wassily Hoeffding, for sums of independent and bounded random variables whose mean is assumed to be known. As a result we can generalise and improve upon this inequality. We show that Hoeffding's bound is optimal in a broader sense. Our approach allows to obtain "missing" factors in Hoeffding's inequality whose existence is motivated by the central limit theorem. The later re…
In this article we present a Bernstein inequality for sums of random variables which are defined on a spatial lattice structure. The inequality can be used to derive concentration inequalities. It can be useful to obtain consistency properties for nonparametric estimators of conditional expectation functions.
We establish a Bernstein-type inequality for a class of stochastic processes that include the classical geometrically $ϕ$-mixing processes, Rio's generalization of these processes, as well as many time-discrete dynamical systems. Modulo a logarithmic factor and some constants, our Bernstein-type inequality coincides with the classical Bernstein inequality for i.i.d.~data. We further use this new Bernstein-type inequality to derive an oracle inequality for generic regularized empirical risk minimization algorithms and data generated by such processes. Applying this oracle inequality to support …
The Entropy method provides a powerful framework for proving scalar concentration inequalities by establishing functional inequalities like Poincare and log-Sobolev inequalities. These inequalities are especially useful for deriving concentration for dependent distributions coming from stationary distributions of Markov chains. In contrast to the scalar case, the study of matrix concentration inequalities is fairly recent and there does not seem to be a general framework for proving matrix concentration inequalities using the Entropy method. In this paper, we initiate the study of Matrix value…
This note contains some estimates for tail spaces on Hamming cube. We use analytic paraproduct operator for that purpose. We also show several types of Bernstein--Markov inequalities for Banach space valued functions on Hamming cube. Here the novelty is in getting rid of some irritating logarithms and in proving Bernstein--Markov inequalities for $|\nabla f|_X$ rather than for $Δ^{1/2} f$ for $X$ valued polynomials on Hamming cube.
We introduce a Bernstein-type inequality which serves to uniformly control quadratic forms of gaussian variables. The latter can for example be used to derive sharp model selection criteria for linear estimation in linear regression and linear inverse problems via penalization, and we do not exclude that its scope of application can be made even broader.
We present a new, very short proof of a conjecture by I. Raşa, which is an inequality involving basic Bernstein polynomials and convex functions. It was affirmed positively very recently by J. Mrowiec, T. Rajba and S. Wąsowicz (2017) by the use of stochastic convex orders, as well as by Abel (2017) who simplified their proof. We give a useful sufficient condition for the verification of some stochastic convex order relations, which in the case of binomial distributions are equivalent to the I. Raşa inequality. We give also the corresponding inequalities for other distributions. Our methods all…
We prove a multivariate version of Hoeffding's inequality about the distribution of homogeneous polynomials of Rademacher functions. The proof is based on such an estimate about the moments of homogeneous polynomials of Rademacher functions which can be considered as an improvement of Borell's inequality in a most important special case.
Since Kilmer et al. introduced the new multiplication method between two third-order tensors around 2008 and third-order tensors with such multiplication structure are also called as T-product tensors, T-product tensors have been applied to many fields in science and engineering, such as low-rank tensor approximation, signal processing, image feature extraction, machine learning, computer vision, and the multi-view clustering problem, etc. However, there are very few works dedicated to exploring the behavior of random T-product tensors. This work considers the problem about the tail behavior o…
We present some extensions of Bernstein's concentration inequality for random matrices. This inequality has become a useful and powerful tool for many problems in statistics, signal processing and theoretical computer science. The main feature of our bounds is that, unlike the majority of previous related results, they do not depend on the dimension $d$ of the ambient space. Instead, the dimension factor is replaced by the "effective rank" associated with the underlying distribution that is bounded from above by $d$. In particular, this makes an extension to the infinite-dimensional setting po…
Using the method of transportation-information inequality introduced in \cite{GLWY}, we establish Bernstein type's concentration inequalities for empirical means $\frac 1t \int_0^t g(X_s)ds$ where $g$ is a unbounded observable of the symmetric Markov process $(X_t)$. Three approaches are proposed : functional inequalities approach ; Lyapunov function method ; and an approach through the Lipschitzian norm of the solution to the Poisson equation. Several applications and examples are studied.
We prove a new Bernstein-type inequality for the log-likelihood function of Bernoulli variables. In contrast to classical Bernstein's inequality and Hoeffding's inequality when applied to the log-likelihood, the new bound is independent of the parameters of the Bernoulli variables and therefore does not blow up as the parameters approach 0 or 1. The new inequality strengthens certain theoretical results on likelihood-based methods for community detection in networks and can be applied to other likelihood-based methods for binary data.
The Bernstein inequality is a tight upper bound on tail probabilities for independent random variables. Freedman extended the Bernstein inequality to martingales with differences bounded from above, and then Dzhaparidze and van Zanten generalized Freedman's result to non-bounded locally square integrable martingales. In this paper, we derive some Dzhaparidze-van Zanten type inequalities for self-normalized martingales with square integrable and non-square integrable differences.
The concentration inequality approach for normal approximation by Stein's method is generalized to the multivariate setting. We use this approach to prove a non-smooth function distance for multivariate normal approximation for standardized sums of $k$-dimensional independent random vectors $W=\sum_{i=1}^n X_i$ with an error bound of order $k^{1/2}γ$ where $γ=\sum_{i=1}^n E|X_i|^3$. For sums of locally dependent (unbounded) random vectors, we obtain a fourth moment bound which is typically of order $O_k(1/\sqrt{n})$, as well as a third moment bound which is typically of order $O_k(\log n/\sqrt…
The Gaussian correlation inequality for multivariate zero-mean normal probabilities of symmetrical n-rectangles can be considered as an inequality for multivariate gamma distributions (in the sense of Krishnamoorthy and Parthasarathy [5]) with one degree of freedom. Its generalization to all integer degrees of freedom and sufficiently large non-integer "degrees of freedom" was recently proved in [10]. Here, this inequality is partly extended to smaller non-integer degrees of freedom and in particular - in a weaker form - to all infinitely divisible multivariate gamma distributions. A further m…
We prove that for any pair of i.i.d. random variables $X,Y$ with finite moment of order $a \in (0,2]$ it is true that $E |X-Y|^a \leq E |X+Y|^a$. Surprisingly, this inequality turns out to be related with bifractional Brownian motion. We extend this result to Bernstein functions and provide some counter-examples.
We obtain a Bernstein type Gaussian concentration inequality for martingales. Our inequality improves the Azuma-Hoeffding inequality for moderate deviations $x$. Following the work of McDiarmid (1989), Talagrand (1996) and Boucheron, Lugosi and Massart (2000,2003), we show that our result can be applied to the concentration of random functions, Erdös-Rényi random graph, and maxima of empirical processes. Several interesting Gaussian concentration inequalities have been obtained.
In this paper, we consider a large class of subordinate random walks $X$ on integer lattice $\mathbb{Z}^d$ via subordinators with Laplace exponents which are complete Bernstein functions satisfying a certain lower scaling condition at zero. We establish estimates for one-step transition probabilities, the Green function and the Green function of a ball, and prove the Harnack inequality for non-negative harmonic functions.
The Hanson-Wright inequality is an upper bound for tails of real quadratic forms in independent subgaussian random variables. In this work, we extend the Hanson-Wright inequality for the maximum eigenvalue of the quadratic sum of random Hermitian tensors under Einstein product. We first prove Weyl inequality for tensors under Einstein product and apply this fact to separate the quadratic form of random Hermitian tensors into diagonal sum and coupling (non-diagonal) sum parts. For the diagonal part, we can apply Bernstein inequality to bound the tail probability of the maximum eigenvalue of the…
Recent development in high-dimensional statistical inference has necessitated concentration inequalities for a broader range of random variables. We focus on sub-Weibull random variables, which extend sub-Gaussian or sub-exponential random variables to allow heavy-tailed distributions. This paper presents concentration inequalities for independent sub-Weibull random variables with finite Generalized Bernstein-Orlicz norms, providing generalized Bernstein's inequalities and Rosenthal-type moment bounds. The tightness of the proposed bounds is shown through lower bounds of the concentration ineq…
Let $\pa{X_{t}}_{t\in T}$ be a family of real-valued centered random variables indexed by a countable set $T$. In the first part of this paper, we establish exponential bounds for the deviation probabilities of the supremum $Z=\sup_{t\in T}X_{t}$ by using the generic chaining device introduced in Talagrand (2005). Compared to concentration-type inequalities, these bounds offer the advantage to hold under weaker conditions on the family $\pa{X_{t}}_{t\in T}$. The second part of the paper is oriented towards statistics. We consider the regression setting $Y=f+\eps$ where $f$ is an unknown vector…
In this paper we prove a parabolic version of the Littlewood-Paley inequality for the operators of the type $ϕ(-Δ)$, where $ϕ$ is a Bernstein function. As an application, we construct an $L_p$-theory for the stochastic integro-differential equations of the type $du=(-ϕ(-Δ)u+f)dt +gdW_t$.
For a multivariate stationary process, we develop explicit representations for the finite predictor coefficient matrices, the finite prediction error covariance matrices and the partial autocorrelation function (PACF) in terms of the Fourier coefficients of its phase function in the spectral domain. The derivation is based on a novel alternating projection technique and the use of the forward and backward innovations corresponding to the predictions based on the infinite past and future, respectively. We show that such representations are ideal for studying the rates of convergence of the fini…
The Gaussian product inequality is an important conjecture concerning the moments of Gaussian random vectors. While all attempts to prove the Gaussian product inequality in full generality have been unsuccessful to date, numerous partial results have been derived in recent decades and we provide here further results on the problem. Most importantly, we establish a strong version of the Gaussian product inequality for multivariate gamma distributions in the case of nonnegative correlations, thereby extending a result recently derived by Genest and Ouimet [5]. Further, we show that the Gaussian …
For a polynomial $P_n$ of degree $n$, Bernstein's inequality states that $\|P_n'\| \le n \|P_n\|$ for all $L^p$ norms on the unit circle, $0<p\le\infty,$ with equality for $P_n(z)= c z^n.$ We study this inequality for random polynomials, and show that the expected (average) and almost sure value of $\Vert P_n' \Vert/\Vert P_n\Vert$ is often different from the classical deterministic upper bound $n$. In particular, for circles of radii less than one, the ratio $\Vert P_n' \Vert/\Vert P_n\Vert$ is almost surely bounded as $n$ tends to infinity, and its expected value is uniformly bounded for all…
Let $μ$ be a Gaussian measure (say, on ${\bf R}^n$) and let $K, L \subset {\bf R}^n$ be such that K is convex, $L$ is a "layer" (i.e. $L = \{x : a \leq < x,u > \leq b \}$ for some $a$, $b \in {\bf R}$ and $u \in {\bf R}^n$) and the centers of mass (with respect to $μ$) of $K$ and $L$ coincide. Then $μ(K \cap L) \geq μ(K) \cdot μ(L)$. This is motivated by the well-known "positive correlation conjecture" for symmetric sets and a related inequality of Sidak concerning confidence regions for means of multivariate normal distributions. The proof uses an apparently hitherto unknown estimate for the …
