In this paper, we derive new, nearly optimal bounds for the Gaussian approximation to scaled averages of $n$ independent high-dimensional centered random vectors $X_1,\dots,X_n$ over the class of rectangles in the case when the covariance matrix of the scaled average is non-degenerate. In the case of bounded $X_i$'s, the implied bound for the Kolmogorov distance between the distribution of the scaled average and the Gaussian vector takes the form $$C (B^2_n \log^3 d/n)^{1/2} \log n,$$ where $d$ is the dimension of the vectors and $B_n$ is a uniform envelope constant on components of $X_i$'s. This bound is sharp in terms of $d$ and $B_n$, and is nearly (up to $\log n$) sharp in terms of the sample size $n$. In addition, we show that similar bounds hold for the multiplier and empirical bootstrap approximations. Moreover, we establish bounds that allow for unbounded $X_i$'s, formulated solely in terms of moments of $X_i$'s. Finally, we demonstrate that the bounds can be further improved in some special smooth and zero-skewness cases.
We provide a functional central limit theorem for a broad class of smooth functions for possibly noncausal multivariate linear processes with time-varying coefficients. Since the limiting processes depend on unknown quantities, we propose a local block bootstrap procedure to circumvent this inconvenience in practical applications. In particular, we prove bootstrap validity for a very broad class of processes. Our results are illustrated by some numerical examples.
The main objective of this paper is to establish bootstrap uniform functional central limit theorem for Harris recurrent Markov chains over uniformly bounded classes of functions. We show that the result can be generalized also to the unbounded case. To avoid some complicated mixing conditions, we make use of the well-known regeneration properties of Markov chains. We show that in the atomic case the proof of the bootstrap uniform central limit theorem for Markov chains for functions dominated by a function in $L^{2}$ space proposed by Radulović (2004) can be significantly simplified. Finally,…
The asymptotic normality of U-statistics has so far been proved for iid data and under various mixing conditions such as absolute regularity, but not for strong mixing. We use a coupling technique introduced in 1983 by Bradley to prove a new generalized covariance inequality similar to Yoshihara's. It follows from the Hoeffding-decomposition and this inequality that U-statistics of strongly mixing observations converge to a normal limit if the kernel of the U-statistic fulfills some moment and continuity conditions. The validity of the bootstrap for U-statistics has until now only been establi…
Non-asymptotic bounds for Gaussian and bootstrap approximation have recently attracted significant interest in high-dimensional statistics. This paper studies Berry-Esseen bounds for such approximations with respect to the multivariate Kolmogorov distance, in the context of a sum of $n$ random vectors that are $p$-dimensional and i.i.d. Up to now, a growing line of work has established bounds with mild logarithmic dependence on $p$. However, the problem of developing corresponding bounds with near $n^{-1/2}$ dependence on $n$ has remained largely unresolved. Within the setting of random vector…
Wasserstein barycenters and variance-like criteria based on the Wasserstein distance are used in many problems to analyze the homogeneity of collections of distributions and structural relationships between the observations. We propose the estimation of the quantiles of the empirical process of Wasserstein's variation using a bootstrap procedure. We then use these results for statistical inference on a distribution registration model for general deformation functions. The tests are based on the variance of the distributions with respect to their Wasserstein's barycenters for which we prove cen…
We will show under very weak conditions on differentiability and dependence that the central limit theorem for quantiles holds and that the block bootstrap is weakly consistent. Under slightly stronger conditions, the bootstrap is strongly consistent. Without the differentiability condition, quantiles might have a non-normal asymptotic distribution and the bootstrap might fail.
This paper deals with the Gaussian and bootstrap approximations to the distribution of the max statistic in high dimensions. This statistic takes the form of the maximum over components of the sum of independent random vectors and its distribution plays a key role in many high-dimensional econometric problems. Using a novel iterative randomized Lindeberg method, the paper derives new bounds for the distributional approximation errors. These new bounds substantially improve upon existing ones and simultaneously allow for a larger class of bootstrap methods.
This paper derives central limit and bootstrap theorems for probabilities that sums of centered high-dimensional random vectors hit hyperrectangles and sparsely convex sets. Specifically, we derive Gaussian and bootstrap approximations for probabilities $\Pr(n^{-1/2}\sum_{i=1}^n X_i\in A)$ where $X_1,\dots,X_n$ are independent random vectors in $\mathbb{R}^p$ and $A$ is a hyperrectangle, or, more generally, a sparsely convex set, and show that the approximation error converges to zero even if $p=p_n\to \infty$ as $n \to \infty$ and $p \gg n$; in particular, $p$ can be as large as $O(e^{Cn^c})$…
New goodness-of-fit tests for Markovian models in time series analysis are developed which are based on the difference between a fully nonparametric estimate of the one-step transition distribution function of the observed process and that of the model class postulated under the null hypothesis. The model specification under the null allows for Markovian models, the transition mechanisms of which depend on an unknown vector of parameters and an unspecified distribution of i.i.d. innovations. Asymptotic properties of the test statistic are derived and the critical values of the test are found u…
We establish a central limit theorem for (a sequence of) multivariate martingales which dimension potentially grows with the length $n$ of the martingale. A consequence of the results are Gaussian couplings and a multiplier bootstrap for the maximum of a multivariate martingale whose dimensionality $d$ can be as large as $e^{n^c}$ for some $c>0$. We also develop new anti-concentration bounds for the maximum component of a high-dimensional Gaussian vector, which we believe is of independent interest. The results are applicable to a variety of settings. We fully develop its use to the estimation…
The problem of testing for the presence of epidemic changes in random fields is investigated. In order to be able to deal with general changes in the marginal distribution, a Cramér-von Mises type test is introduced which is based on Hilbert space theory. A functional central limit theorem for $ρ$-mixing Hilbert space valued random fields is proven. In order to avoid the estimation of the long-run variance and obtain critical values, Shao's dependent wild bootstrap method is adapted to this context. For this, a joint functional central limit theorem for the original and the bootstrap sample is…
Statistical methods for functional data are of interest for many applications. In this paper, we prove a central limit theorem for random variables taking their values in a Hilbert space. The random variables are assumed to be weakly dependent in the sense of near epoch dependence, where the underlying process fulfills some mixing conditions. As parametric inference in an infinite dimensional space is difficult, we show that the nonoverlapping block bootstrap is consistent. Furthermore, we show how these results can be used for degenerate von Mises-statistics.
A new test for structural changes in functional data is investigated. It is based on Hilbert space theory and critical values are deduced from bootstrap iterations. Thus a new functional central limit theorem for the block bootstrap in a Hilbert space is required. The test can also be used to detect changes in the marginal distribution of random vectors, which is supplemented by a simulation study. Our methods are applied to hydrological data from Germany.
Improved Central Limit Theorem and Bootstrap Approximations for Linear Stochastic Approximation
HSE University（俄罗斯莫斯科高等经济学院）；Ecole Polytechnique（巴黎高等理工学院）；Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）；Shenzhen International Center of Mathematics, Southern University of Science and Technology（深圳国际数学中心，南方科技大学）
This article reviews recent progress in high-dimensional bootstrap. We first review high-dimensional central limit theorems for distributions of sample mean vectors over the rectangles, bootstrap consistency results in high dimensions, and key techniques used to establish those results. We then review selected applications of high-dimensional bootstrap: construction of simultaneous confidence sets for high-dimensional vector parameters, multiple hypothesis testing via stepdown, post-selection inference, intersection bounds for partially identified parameters, and inference on best policies in …
This paper is mainly concerned with asymptotic studies of weighted bootstrap for u- and v-statistics. We derive the consistency of the weighted bootstrap u- and v-statistics, based on i.i.d. and non i.i.d. observations, from some more general results which we first establish for sums of randomly weighted arrays of random variables. Some of the results in this paper significantly extend some well-known results on consistency of u-statistics and also consistency of sums of arrays of random variables. We also employ a new approach to conditioning to derive a conditional CLT for weighted bootstrap…
We derive strong approximations to the supremum of the non-centered empirical process indexed by a possibly unbounded VC-type class of functions by the suprema of the Gaussian and bootstrap processes. The bounds of these approximations are non-asymptotic, which allows us to work with classes of functions whose complexity increases with the sample size. The construction of couplings is not of the Hungarian type and is instead based on the Slepian-Stein methods and Gaussian comparison inequalities. The increasing complexity of classes of functions and non-centrality of the processes make the res…
We propose a distributed bootstrap method for simultaneous inference on high-dimensional massive data that are stored and processed with many machines. The method produces an $\ell_\infty$-norm confidence region based on a communication-efficient de-biased lasso, and we propose an efficient cross-validation approach to tune the method at every iteration. We theoretically prove a lower bound on the number of communication rounds $τ_{\min}$ that warrants the statistical accuracy and efficiency. Furthermore, $τ_{\min}$ only increases logarithmically with the number of workers and the intrinsic di…
The theory for multiplier empirical processes has been one of the central topics in the development of the classical theory of empirical processes, due to its wide applicability to various statistical problems. In this paper, we develop theory and tools for studying multiplier $U$-processes, a natural higher-order generalization of the multiplier empirical processes. To this end, we develop a multiplier inequality that quantifies the moduli of continuity of the multiplier $U$-process in terms of that of the (decoupled) symmetrized $U$-process. The new inequality finds a variety of applications…
We introduce a bootstrap procedure for high-frequency statistics of Brownian semistationary processes. More specifically, we focus on a hypothesis test on the roughness of sample paths of Brownian semistationary processes, which uses an estimator based on a ratio of realized power variations. Our new resampling method, the local fractional bootstrap, relies on simulating an auxiliary fractional Brownian motion that mimics the fine properties of high frequency differences of the Brownian semistationary process under the null hypothesis. We prove the first order validity of the bootstrap method …
One of the most widely applied unit root test, Phillips-Perron test, enjoys in general highpowers, but suffers from size distortions when moving average noise exists. As a remedy, thispaper proposes a nonparametric bootstrap unit root test that specifically targets moving aver-age noise. Via a bootstrap functional central limit theorem, the consistency of this bootstrapapproach is established under general assumptions which allows a large family of non-linear timeseries. In simulation, this bootstrap test alleviates the size distortions of the Phillips-Perrontest while preserving its high powe…
In recent years, bootstrap methods have drawn attention for their ability to approximate the laws of "max statistics" in high-dimensional problems. A leading example of such a statistic is the coordinate-wise maximum of a sample average of $n$ random vectors in $\mathbb{R}^p$. Existing results for this statistic show that the bootstrap can work when $n\ll p$, and rates of approximation (in Kolmogorov distance) have been obtained with only logarithmic dependence in $p$. Nevertheless, one of the challenging aspects of this setting is that established rates tend to scale like $n^{-1/6}$ as a func…
We consider the problem of quantifying uncertainty for the estimation error of the leading eigenvector from Oja's algorithm for streaming principal component analysis, where the data are generated IID from some unknown distribution. By combining classical tools from the U-statistics literature with recent results on high-dimensional central limit theorems for quadratic forms of random vectors and concentration of matrix products, we establish a weighted $χ^2$ approximation result for the $\sin^2$ error between the population eigenvector and the output of Oja's algorithm. Since estimating the c…
Particle island models (Vergé et al., 2013) provide a means of parallelization of sequential Monte Carlo methods, and in this paper we present novel convergence results for algorithms of this sort. In particular we establish a central limit theorem - as the number of islands and the common size of the islands tend jointly to infinity - of the double bootstrap algorithm with possibly adaptive selection on the island level. For this purpose we introduce a notion of archipelagos of weighted islands and find conditions under which a set of convergence properties are preserved by different operatio…
We consider bootstrap-based testing for threshold effects in non-linear threshold autoregressive (TAR) models. It is well-known that classic tests based on asymptotic theory tend to be oversized in the case of small, or even moderate sample sizes, or when the estimated parameters indicate non-stationarity, as often witnessed in the analysis of financial or climate data. To address the issue we propose a supremum Lagrange Multiplier test statistic (sLMb), where the null hypothesis specifies a linear autoregressive (AR) model against the alternative of a TAR model. We consider a recursive bootst…
Kronecker product covariance structure provides an efficient way to modeling the inter-correlations of matrix-variate data. In this paper, we propose testing statistics for Kronecker product covariance matrix based on linear spectral statistics of renormalized sample covariance matrices. Central limit theorem is proved for the linear spectral statistics with explicit formulas for mean and covariance functions, which fills the gap in the literature. We then theoretically justify that the proposed testing statistics have well-controlled sizes and strong powers. To facilitate practical usefulness…
Statistics derived from the eigenvalues of sample covariance matrices are called spectral statistics, and they play a central role in multivariate testing. Although bootstrap methods are an established approach to approximating the laws of spectral statistics in low-dimensional problems, these methods are relatively unexplored in the high-dimensional setting. The aim of this paper is to focus on linear spectral statistics as a class of prototypes for developing a new bootstrap in high-dimensions --- and we refer to this method as the Spectral Bootstrap. In essence, the method originates from t…
We study the residual bootstrap (RB) method in the context of high-dimensional linear regression. Specifically, we analyze the distributional approximation of linear contrasts $c^{\top} (\hatβ_ρ-β)$, where $\hatβ_ρ$ is a ridge-regression estimator. When regression coefficients are estimated via least squares, classical results show that RB consistently approximates the laws of contrasts, provided that $p\ll n$, where the design matrix is of size $n\times p$. Up to now, relatively little work has considered how additional structure in the linear model may extend the validity of RB to the settin…
When randomized ensemble methods such as bagging and random forests are implemented, a basic question arises: Is the ensemble large enough? In particular, the practitioner desires a rigorous guarantee that a given ensemble will perform nearly as well as an ideal infinite ensemble (trained on the same data). The purpose of the current paper is to develop a bootstrap method for solving this problem in the context of regression --- which complements our companion paper in the context of classification (Lopes 2019). In contrast to the classification setting, the current paper shows that theoretica…
We present a new approach to the bootstrap for chains of infinite order taking values on a finite alphabet. It is based on a sequential Bootstrap Central Limit Theorem for the sequence of canonical Markov approximations of the chain of infinite order. Combined with previous results on the rate of approximation this leads to a Central Limit Theorem for the bootstrapped estimator of the sample mean which is the main result of this paper.
We unconditionally prove a central limit theorem for linear statistics of the zeros of the Riemann zeta function with diverging variance. Previously, theorems of this sort have been proved under the assumption of the Riemann hypothesis. The result mirrors central limit theorems in random matrix theory that have been proved by Szegő, Spohn, and Soshnikov among others, and therefore provides support for the view that the zeros of the zeta function are distributed like the eigenvalues of a random matrix. A key ingredient in our proof is a simple bootstrapping of classical zero density estimates o…
This article presents a bootstrap approximation to the Lp_statistics of kernel density estimator in length-biased model. Length-biased data arise in many situations, such as survival analysis, renewal processes and physics. The article establishes a bootstrap central limit theorem for the corresponding bootstrap version of this Lp_statistic. The bootstrap is a widely used tool in statistics and, therefore, the properties of this bootstrap approximation are of great interest in applied as well as in theoretical statistics.
