Let $\pa{X_{t}}_{t\in T}$ be a family of real-valued centered random variables indexed by a countable set $T$. In the first part of this paper, we establish exponential bounds for the deviation probabilities of the supremum $Z=\sup_{t\in T}X_{t}$ by using the generic chaining device introduced in Talagrand (2005). Compared to concentration-type inequalities, these bounds offer the advantage to hold under weaker conditions on the family $\pa{X_{t}}_{t\in T}$. The second part of the paper is oriented towards statistics. We consider the regression setting $Y=f+\eps$ where $f$ is an unknown vector of $\R^{n}$ and $\eps$ is a random vector the components of which are independent, centered and admit finite Laplace transforms in a neighborhood of 0. Our aim is to estimate $f$ from the observation of $Y$ by mean of a model selection approach among a collection of linear subspaces of $\R^{n}$. The selection procedure we propose is based on the minimization of a penalized criterion the penalty of which is calibrated by using the deviation bounds established in the first part of this paper. More precisely, we study suprema of random variables of the form $X_{t}=\sum_{i=1}^{n}t_{i}\eps_{i}$ when $t$ varies among the unit ball of a linear subspace of $\R^{n}$. We finally show that our estimator satisfies some oracle-type inequality under suitable assumptions on the metric structures of the linear spaces of the collection.
We use the generic chaining device proposed by Talagrand to establish exponential bounds on the deviation probability of some suprema of random processes. Then, given a random vector $ξ$ in $\R^{n}$ the components of which are independent and admit a suitable exponential moment, we deduce a deviation inequality for the squared Euclidean norm of the projection of $ξ$ onto a linear subspace of $\R^{n}$. Finally, we provide an application of such an inequality to statistics, performing model selection in the regression setting when the errors are possibly non-Gaussian and the collection of models…
In this article we present a Bernstein inequality for sums of random variables which are defined on a spatial lattice structure. The inequality can be used to derive concentration inequalities. It can be useful to obtain consistency properties for nonparametric estimators of conditional expectation functions.
We establish a Bernstein-type inequality for a class of stochastic processes that include the classical geometrically $ϕ$-mixing processes, Rio's generalization of these processes, as well as many time-discrete dynamical systems. Modulo a logarithmic factor and some constants, our Bernstein-type inequality coincides with the classical Bernstein inequality for i.i.d.~data. We further use this new Bernstein-type inequality to derive an oracle inequality for generic regularized empirical risk minimization algorithms and data generated by such processes. Applying this oracle inequality to support …
We present some extensions of Bernstein's concentration inequality for random matrices. This inequality has become a useful and powerful tool for many problems in statistics, signal processing and theoretical computer science. The main feature of our bounds is that, unlike the majority of previous related results, they do not depend on the dimension $d$ of the ambient space. Instead, the dimension factor is replaced by the "effective rank" associated with the underlying distribution that is bounded from above by $d$. In particular, this makes an extension to the infinite-dimensional setting po…
We introduce a Bernstein-type inequality which serves to uniformly control quadratic forms of gaussian variables. The latter can for example be used to derive sharp model selection criteria for linear estimation in linear regression and linear inverse problems via penalization, and we do not exclude that its scope of application can be made even broader.
In this article we present a Bernstein inequality for sums of random variables which are defined on a graphical network whose nodes grow at an exponential rate. The inequality can be used to derive concentration inequalities in highly-connected networks. It can be useful to obtain consistency properties for nonparametric estimators of conditional expectation functions which are derived from such networks.
We show somewhat unexpectedly that whenever a general Bernstein-type maximal inequality holds for partial sums of a sequence of random variables, a maximal form of the inequality is also valid.
We prove a new Bernstein-type inequality for the log-likelihood function of Bernoulli variables. In contrast to classical Bernstein's inequality and Hoeffding's inequality when applied to the log-likelihood, the new bound is independent of the parameters of the Bernoulli variables and therefore does not blow up as the parameters approach 0 or 1. The new inequality strengthens certain theoretical results on likelihood-based methods for community detection in networks and can be applied to other likelihood-based methods for binary data.
Recent development in high-dimensional statistical inference has necessitated concentration inequalities for a broader range of random variables. We focus on sub-Weibull random variables, which extend sub-Gaussian or sub-exponential random variables to allow heavy-tailed distributions. This paper presents concentration inequalities for independent sub-Weibull random variables with finite Generalized Bernstein-Orlicz norms, providing generalized Bernstein's inequalities and Rosenthal-type moment bounds. The tightness of the proposed bounds is shown through lower bounds of the concentration ineq…
This paper develops a new direct approach to approximating suprema of general empirical processes by a sequence of suprema of Gaussian processes, without taking the route of approximating whole empirical processes in the sup-norm. We prove an abstract approximation theorem applicable to a wide variety of statistical problems, such as construction of uniform confidence bands for functions. Notably, the bound in the main approximation theorem is nonasymptotic and the theorem allows for functions that index the empirical process to be unbounded and have entropy divergent with the sample size. The…
Initially motivated by the study of the non-asymptotic properties of non-parametric tests based on permutation methods, concentration inequalities for uniformly permuted sums have been largely studied in the literature. Recently, Delyon et al. proved a new Bernstein-type concentration inequality based on martingale theory. This work presents a new proof of this inequality based on the fundamental inequalities for random permutations of Talagrand. The idea is to first obtain a rough inequality for the square root of the permuted sum, and then, iterate the previous analysis and plug this first i…
We present Hoeffding-type and Bernstein-type inequalities for right-censored data. The inequalities bound the difference between an inverse of the probability of censoring weighting (IPCW) estimator and its expectation. We first discuss the asymptotic properties of the estimator and provide conditions for its efficiency. We present standard, data dependent, and uniform Hoeffding-type inequalities. We then present a Bernstein-type inequality. Finally, we show how to apply these inequalities in an empirical risk minimization setting.
A concentration result for quadratic form of independent subgaussian random variables is derived. If the moments of the random variables satisfy a "Bernstein condition", then the variance term of the Hanson-Wright inequality can be improved. The Bernstein condition is satisfied, for instance, by all log-concave subgaussian distributions.
As an alternative to the well-known methods of "chaining" and "bracketing" that have been developed in the study of random fields, a new method, which is based on a stochastic maximal inequality derived by using Itô's formula and on a new concept named strict countability, is presented. The main results are some weak convergence theorems for sequences of separable random fields of locally square-integrable martingales under the uniform topology with the help also of entropy methods. As special cases, some new results for i.i.d. random sequences, including a new Donsker theorem and a moment bou…
We prove a new and general concentration inequality for the excess risk in least-squares regression with random design and heteroscedastic noise. No specific structure is required on the model, except the existence of a suitable function that controls the local suprema of the empirical process. So far, only the case of linear contrast estimation was tackled in the literature with this level of generality on the model. We solve here the case of a quadratic contrast, by separating the behavior of a linearized empirical process and the empirical process driven by the squares of functions of model…
Recently, Chernozhukov, Chetverikov, and Kato [Ann. Statist. 42 (2014) 1564--1597] developed a new Gaussian comparison inequality for approximating the suprema of empirical processes. This paper exploits this technique to devise sharp inference on spectra of large random matrices. In particular, we show that two long-standing problems in random matrix theory can be solved: (i) simple bootstrap inference on sample eigenvalues when true eigenvalues are tied; (ii) conducting two-sample Roy's covariance test in high dimensions. To establish the asymptotic results, a generalized $ε$-net argument re…
In this paper we prove exponential inequalities (also called Bernstein's inequality) for fractional martingales. As an immediate corollary, we will discuss weak law of large numbers for fractional martingales under divergence assumption on the $β-$variation of the fractional martingale. A non trivial example of application of this convergence result is proposed.
Incomplete U-statistics have been proposed to accelerate computation. They use only a subset of the subsamples required for kernel evaluations by complete U-statistics. This paper gives a finite sample bound in the style of Bernstein's inequality. Applied to complete U-statistics the resulting inequality improves over the bounds of both Hoeffding and Arcones. For randomly determined subsamples it is shown, that, as soon as the their number reaches the square of the sample-size, the same order bound is obtained as for the complete statistic.
For every given real value of the ratio $μ:=A_X/G_X>1$ of the arithmetic and geometric means of a positive random variable $X$ and every real $v>0$, exact upper bounds on the right- and left-tail probabilities $\mathsf{P}(X/G_X\ge v)$ and $\mathsf{P}(X/G_X\le v)$ are obtained, in terms of $μ$ and $v$. In particular, these bounds imply that $X/G_X\to1$ in probability as $A_X/G_X\downarrow1$. Such a result may be viewed as a converse to a reverse Jensen inequality for the strictly concave function $f=\ln$, whereas the well-known Cantelli and Chebyshev inequalities may be viewed as converses to a…
We obtain a Bernstein-type inequality for sums of Banach-valued random variables satisfying a weak dependence assumption of general type and under certain smoothness assumptions of the underlying Banach norm. We use this inequality in order to investigate in the asymptotical regime the error upper bounds for the broad family of spectral regularization methods for reproducing kernel decision rules, when trained on a sample coming from a $τ-$mixing process.
