In this paper we present a tail inequality for the maximum of partial sums of a weakly dependent sequence of random variables that are not necessarily bounded. The class considered includes geometrically and subgeometrically strongly mixing sequences. The result is then used to derive asymptotic moderate deviations results. Applications include classes of Markov chains, functions of linear processes with absolutely regular innovations and ARCH models
We modify the classical Bernstein's inequality for the sums of independent centered random variables (r.v.) in the terms of relative tails or moments. We built also some examples in order to show the exactness of offered results.
In this paper we obtain a Bernstein type inequality for the sum of self-adjoint centered and geometrically absolutely regular random matrices with bounded largest eigenvalue. This inequality can be viewed as an extension to the matrix setting of the Bernstein-type inequality obtained by Merlevède et al. (2009) in the context of real-valued bounded random variables that are geometrically absolutely regular. The proofs rely on decoupling the Laplace transform of a sum on a Cantor-like set of random matrices.
In this paper we obtain a Bernstein type inequality for a class of weakly dependent and bounded random variables. The proofs lead to a moderate deviations principle for sums of bounded random variables with exponential decay of the strong mixing coeficients that complements the large deviation result obtained by Bryc and Dembo (1998) under superexponential mixing rates.
We give a distribution-dependent concentration inequality for functions of independent variables. The result extends Bernstein's inequality from sums to more general functions, whose variation in any argument does not depend too much on the other arguments. Applications sharpen existing bounds for U-statistics and the generalization error of regularized least squares.
We prove a multivariate version of Bernstein's inequality about the probability that degenerate $U$-statistics take a value larger than some number $u$. This is an improvement of former estimates for the same problem which yields an asymptotically sharp estimate for not too large numbers $u$. This paper also contains an analogous bound about the distribution of multiple Wiener-Ito integrals. Their comparison shows that our results are sharp. The proofs are based on good estimates about high moments of multiple random integrals. They are obtained by means of a diagram formula which enables us t…
Using the renewal approach we prove Bernstein-like inequalities for additive functionals of geometrically ergodic Markov chains, thus obtaining counterparts of inequalities for sums of independent random variables. The coefficient in the sub-Gaussian part of our estimate is the asymptotic variance of the additive functional, i.e. the variance of the limiting Gaussian variable in the Central Limit Theorem for Markov chains. This refines earlier results by R. Adamczak and W. Bednorz, which were obtained under the additional assumption of strong aperiodicity of the chain.
We consider the three dimensional array $\mathcal{A} = \{a_{i,j,k}\}_{1\le i,j,k \le n}$, with $a_{i,j,k} \in [0,1]$, and the two random statistics $T_{1}:= \sum_{i=1}^n \sum_{j=1}^n a_{i,j,σ(i)}$ and $T_{2}:= \sum_{i=1}^{n} a_{i,σ(i),π(i)}$, where $σ$ and $π$ are chosen independently from the set of permutations of $\{1,2,\ldots,n \}.$ These can be viewed as natural three dimensional generalizations of the statistic $T_{3}=\sum_{i=1}^{n} a_{i,σ(i)}$, considered by Hoeffding \cite{Hoe51}. Here we give Bernstein type concentration inequalities for $T_{1}$ and $T_{2}$ by extending the argument f…
We show that the Bernstein-Hoeffding method can be employed to a larger class of generalized moments. This class includes the exponential moments whose properties play a key role in the proof of a well-known inequality of Wassily Hoeffding, for sums of independent and bounded random variables whose mean is assumed to be known. As a result we can generalise and improve upon this inequality. We show that Hoeffding's bound is optimal in a broader sense. Our approach allows to obtain "missing" factors in Hoeffding's inequality whose existence is motivated by the central limit theorem. The later re…
In this article we present a Bernstein inequality for sums of random variables which are defined on a spatial lattice structure. The inequality can be used to derive concentration inequalities. It can be useful to obtain consistency properties for nonparametric estimators of conditional expectation functions.
We establish a Bernstein-type inequality for a class of stochastic processes that include the classical geometrically $ϕ$-mixing processes, Rio's generalization of these processes, as well as many time-discrete dynamical systems. Modulo a logarithmic factor and some constants, our Bernstein-type inequality coincides with the classical Bernstein inequality for i.i.d.~data. We further use this new Bernstein-type inequality to derive an oracle inequality for generic regularized empirical risk minimization algorithms and data generated by such processes. Applying this oracle inequality to support …
We consider the stochastic integrals of multivariate point processes and study their concentration phenomena. In particular, we obtain a Bernstein type of concentration inequality through Doléans-Dade exponential formula and a uniform exponential inequality using a generic chaining argument. As applications, we obtain a upper bound for a sequence of discrete time martingales indexed by a class of functionals, and so derive the rate of convergence for nonparametric maximum likelihood estimators, which is an improvement of earlier work of van de Geer.
The Entropy method provides a powerful framework for proving scalar concentration inequalities by establishing functional inequalities like Poincare and log-Sobolev inequalities. These inequalities are especially useful for deriving concentration for dependent distributions coming from stationary distributions of Markov chains. In contrast to the scalar case, the study of matrix concentration inequalities is fairly recent and there does not seem to be a general framework for proving matrix concentration inequalities using the Entropy method. In this paper, we initiate the study of Matrix value…
This note contains some estimates for tail spaces on Hamming cube. We use analytic paraproduct operator for that purpose. We also show several types of Bernstein--Markov inequalities for Banach space valued functions on Hamming cube. Here the novelty is in getting rid of some irritating logarithms and in proving Bernstein--Markov inequalities for $|\nabla f|_X$ rather than for $Δ^{1/2} f$ for $X$ valued polynomials on Hamming cube.
We introduce a Bernstein-type inequality which serves to uniformly control quadratic forms of gaussian variables. The latter can for example be used to derive sharp model selection criteria for linear estimation in linear regression and linear inverse problems via penalization, and we do not exclude that its scope of application can be made even broader.
We present a new, very short proof of a conjecture by I. Raşa, which is an inequality involving basic Bernstein polynomials and convex functions. It was affirmed positively very recently by J. Mrowiec, T. Rajba and S. Wąsowicz (2017) by the use of stochastic convex orders, as well as by Abel (2017) who simplified their proof. We give a useful sufficient condition for the verification of some stochastic convex order relations, which in the case of binomial distributions are equivalent to the I. Raşa inequality. We give also the corresponding inequalities for other distributions. Our methods all…
We present some extensions of Bernstein's concentration inequality for random matrices. This inequality has become a useful and powerful tool for many problems in statistics, signal processing and theoretical computer science. The main feature of our bounds is that, unlike the majority of previous related results, they do not depend on the dimension $d$ of the ambient space. Instead, the dimension factor is replaced by the "effective rank" associated with the underlying distribution that is bounded from above by $d$. In particular, this makes an extension to the infinite-dimensional setting po…
Using the method of transportation-information inequality introduced in \cite{GLWY}, we establish Bernstein type's concentration inequalities for empirical means $\frac 1t \int_0^t g(X_s)ds$ where $g$ is a unbounded observable of the symmetric Markov process $(X_t)$. Three approaches are proposed : functional inequalities approach ; Lyapunov function method ; and an approach through the Lipschitzian norm of the solution to the Poisson equation. Several applications and examples are studied.
We prove a new Bernstein-type inequality for the log-likelihood function of Bernoulli variables. In contrast to classical Bernstein's inequality and Hoeffding's inequality when applied to the log-likelihood, the new bound is independent of the parameters of the Bernoulli variables and therefore does not blow up as the parameters approach 0 or 1. The new inequality strengthens certain theoretical results on likelihood-based methods for community detection in networks and can be applied to other likelihood-based methods for binary data.
The Bernstein inequality is a tight upper bound on tail probabilities for independent random variables. Freedman extended the Bernstein inequality to martingales with differences bounded from above, and then Dzhaparidze and van Zanten generalized Freedman's result to non-bounded locally square integrable martingales. In this paper, we derive some Dzhaparidze-van Zanten type inequalities for self-normalized martingales with square integrable and non-square integrable differences.
We obtain a Bernstein type Gaussian concentration inequality for martingales. Our inequality improves the Azuma-Hoeffding inequality for moderate deviations $x$. Following the work of McDiarmid (1989), Talagrand (1996) and Boucheron, Lugosi and Massart (2000,2003), we show that our result can be applied to the concentration of random functions, Erdös-Rényi random graph, and maxima of empirical processes. Several interesting Gaussian concentration inequalities have been obtained.
We obtain moderate deviations theorems and exponential (Bernstein type) concentration inequalities for "nonconventional" sums of the form $S_N=\sum_{n=1}^N (F(ξ_{q_1(n)},ξ_{q_2(n)},...,ξ_{q_\ell(n)})-\bar F)$.
In the present paper we prove that the probabilities of the Pólya urn distribution (with negative replacement) satisfy a monotonicity property similar to that of the binomial distribution (Pólya urn distribution with no replacement). As a consequence, we show that the random variables with Pólya urn distribution (with negative replacement) are stochastically ordered with respect to the parameter giving the initial distribution of the urn. An equivalent formulation of this result shows that the new Bernstein operator recently introduced in [3] is a monotone operator. The proofs are probabilisti…
We introduce two new concepts designed for the study of empirical processes. First, we introduce a new Orlicz norm which we call the Bernstein-Orlicz norm. This new norm interpolates sub-Gaussian and sub-exponential tail behavior. In particular, we show how this norm can be used to simplify the derivation of deviation inequalities for suprema of collections of random variables. Secondly, we introduce chaining and generic chaining along a tree. These simplify the well-known concepts of chaining and generic chaining. The supremum of the empirical process is then studied as a special case. We sho…
In this paper we study the regularity of paths in terms of properties of admissible nets. We show the right concentration inequality above the modulus of continuity. Using the approach we prove the Bernstein type inequality for the empirical processes. Therefore we obtain the best form of concentration for processes studied recently by Mendelson and Paouris. Results of this type are of importance in the compressed sensing theory.
Let $\pa{X_{t}}_{t\in T}$ be a family of real-valued centered random variables indexed by a countable set $T$. In the first part of this paper, we establish exponential bounds for the deviation probabilities of the supremum $Z=\sup_{t\in T}X_{t}$ by using the generic chaining device introduced in Talagrand (2005). Compared to concentration-type inequalities, these bounds offer the advantage to hold under weaker conditions on the family $\pa{X_{t}}_{t\in T}$. The second part of the paper is oriented towards statistics. We consider the regression setting $Y=f+\eps$ where $f$ is an unknown vector…
In this paper we prove a parabolic version of the Littlewood-Paley inequality for the operators of the type $ϕ(-Δ)$, where $ϕ$ is a Bernstein function. As an application, we construct an $L_p$-theory for the stochastic integro-differential equations of the type $du=(-ϕ(-Δ)u+f)dt +gdW_t$.
Freedman's inequality is a martingale counterpart to Bernstein's inequality. This result shows that the large-deviation behavior of a martingale is controlled by the predictable quadratic variation and a uniform upper bound for the martingale difference sequence. Oliveira has recently established a natural extension of Freedman's inequality that provides tail bounds for the maximum singular value of a matrix-valued martingale. This note describes a different proof of the matrix Freedman inequality that depends on a deep theorem of Lieb from matrix analysis. This argument delivers sharp constan…
The maximal inequalities for diffusion processes have drawn increasing attention in recent years. However, the existing proof of the $L^p$ maximum inequalities for the Ornstein-Uhlenbeck process was dubious. Here we give a rigorous proof of the moderate maximum inequalities for the Ornstein-Uhlenbeck process, which include the $L^p$ maximum inequalities as special cases and generalize the remarkable $L^1$ maximum inequalities obtained by Graversen and Peskir [P. Am. Math. Soc., 128(10):3035-3041, 2000]. As a corollary, we also obtain a new moderate maximal inequality for continuous local marti…
Initially motivated by the study of the non-asymptotic properties of non-parametric tests based on permutation methods, concentration inequalities for uniformly permuted sums have been largely studied in the literature. Recently, Delyon et al. proved a new Bernstein-type concentration inequality based on martingale theory. This work presents a new proof of this inequality based on the fundamental inequalities for random permutations of Talagrand. The idea is to first obtain a rough inequality for the square root of the permuted sum, and then, iterate the previous analysis and plug this first i…
A Bernstein-type exponential inequality for (generalized) canonical U-statistics of order 2 is obtained and the Rosenthal and Hoffmann-Jørgensen inequalities for sums of independent random variables are extended to (generalized) U-statistics of any order whose kernels are either nonnegative or canonical
Let $S_N$ be the sum of vector-valued functions defined on a finite Markov chain. An analogue of the Bernstein--Hoeffding inequality is derived for the probability of large deviations of $S_N$ and relates the probability to the spectral gap of the Markov chain. Examples suggest that this inequality is better than alternative inequalities if the chain has a sufficiently large spectral gap and the function is high-dimensional.
In this paper we prove exponential inequalities (also called Bernstein's inequality) for fractional martingales. As an immediate corollary, we will discuss weak law of large numbers for fractional martingales under divergence assumption on the $β-$variation of the fractional martingale. A non trivial example of application of this convergence result is proposed.
For every given real value of the ratio $μ:=A_X/G_X>1$ of the arithmetic and geometric means of a positive random variable $X$ and every real $v>0$, exact upper bounds on the right- and left-tail probabilities $\mathsf{P}(X/G_X\ge v)$ and $\mathsf{P}(X/G_X\le v)$ are obtained, in terms of $μ$ and $v$. In particular, these bounds imply that $X/G_X\to1$ in probability as $A_X/G_X\downarrow1$. Such a result may be viewed as a converse to a reverse Jensen inequality for the strictly concave function $f=\ln$, whereas the well-known Cantelli and Chebyshev inequalities may be viewed as converses to a…
In this paper we extend the Bernstein, Prohorov and Bennett inequalities to the noncommutative setting. In addition we provide an improved version of the noncommutative Rosenthal inequality, essentially due to Nagaev, Pinelis and Pinelis, Utev for commutative random variables. We also present new best constants in Rosenthal's inequality. Applying these results to random Fourier projections, we recover and elaborate on fundamental results from compressed sensing, due to Candes, Romberg and Tao.
