We develop and implement a novel fast bootstrap for dependent data. Our scheme is based on the i.i.d. resampling of the smoothed moment indicators. We characterize the class of parametric and semi-parametric estimation problems for which the method is valid. We show the asymptotic refinements of the proposed procedure, proving that it is higher-order correct under mild assumptions on the time series, the estimating functions, and the smoothing kernel. We illustrate the applicability and the advantages of our procedure for Generalized Empirical Likelihood estimation. As a by-product, our fast bootstrap provides higher-order correct asymptotic confidence distributions. Monte Carlo simulations on an autoregressive conditional duration model provide numerical evidence that the novel bootstrap yields higher-order accurate confidence intervals. A real-data application on dynamics of trading volume of stocks illustrates the advantage of our method over the routinely-applied first-order asymptotic theory, when the underlying distribution of the test statistic is skewed or fat-tailed.
In this paper we study the problem of constructing bootstrap confidence intervals for the Lévy density of the driving Lévy process based on high-frequency observations of a Lévy-driven moving average processes. Using a spectral estimator of the Lévy density, we propose a novel implementations of multiplier and empirical bootstraps to construct confidence bands on a compact set away from the origin. We also provide conditions under which the confidence bands are asymptotically valid.
The problem of quantifying uncertainty about the locations of multiple change points by means of confidence intervals is addressed. The asymptotic distribution of the change point estimators obtained as the local maximisers of moving sum statistics is derived, where the limit distributions differ depending on whether the corresponding size of changes is local, i.e. tends to zero as the sample size increases, or fixed. A bootstrap procedure for confidence interval generation is proposed which adapts to the unknown magnitude of changes and guarantees asymptotic validity both for local and fixed …
One of the most widely applied unit root test, Phillips-Perron test, enjoys in general highpowers, but suffers from size distortions when moving average noise exists. As a remedy, thispaper proposes a nonparametric bootstrap unit root test that specifically targets moving aver-age noise. Via a bootstrap functional central limit theorem, the consistency of this bootstrapapproach is established under general assumptions which allows a large family of non-linear timeseries. In simulation, this bootstrap test alleviates the size distortions of the Phillips-Perrontest while preserving its high powe…
We consider the issue of performing accurate small sample inference in beta autoregressive moving average model, which is useful for modeling and forecasting continuous variables that assumes values in the interval $(0,1)$. The inferences based on conditional maximum likelihood estimation have good asymptotic properties, but their performances in small samples may be poor. This way, we propose bootstrap bias corrections of the point estimators and different bootstrap strategies for confidence interval improvements. Our Monte Carlo simulations show that finite sample inference based on bootstra…
This paper investigates bootstrap-based bias correction of semiparametric estimators of the long memory parameter, $d$, in fractionally integrated processes. The re-sampling method involves the application of the sieve bootstrap to data pre-filtered by a preliminary semiparametric estimate of the long memory parameter. Theoretical justification for using the bootstrap technique to bias adjust log periodogram and semiparametric local Whittle estimators of the memory parameter is provided in the case where the true value of $d$ lies in the range $0\leq d<0.5$. That the bootstrap method provides …
Consider the multiple linear regression model $y_{i} = \boldsymbol{x}'_{i} \boldsymbolβ + ε_{i}$, where $ε_i$'s are independent and identically distributed random variables, $\mathbf{x}_i$'s are known design vectors and $\boldsymbolβ$ is the $p \times 1$ vector of parameters. An effective way of approximating the distribution of the M-estimator $\boldsymbol{\barβ}_n$, after proper centering and scaling, is the Perturbation Bootstrap Method. In this current work, second order results of this non-naive bootstrap method have been investigated. Second order correctness is important for reducing th…
This paper investigates the use of bootstrap-based bias correction of semi-parametric estimators of the long memory parameter in fractionally integrated processes. The re-sampling method involves the application of the sieve bootstrap to data pre-filtered by a preliminary semi-parametric estimate of the long memory parameter. Theoretical justification for using the bootstrap techniques to bias adjust log-periodogram and semi-parametric local Whittle estimators of the memory parameter is provided. Simulation evidence comparing the performance of the bootstrap bias correction with analytical bia…
The inflated beta regression model aims to enable the modeling of responses in the intervals $(0,1]$, $[0,1)$ or $[0,1]$. In this model, hypothesis testing is often performed based on the likelihood ratio statistic. The critical values are obtained from asymptotic approximations, which may lead to distortions of size in small samples. In this sense, this paper proposes the bootstrap Bartlett correction to the statistic of likelihood ratio in the inflated beta regression model. The proposed adjustment only requires a simple Monte Carlo simulation. Through extensive Monte Carlo simulations the f…
In assessing prediction accuracy of multivariable prediction models, optimism corrections are essential for preventing biased results. However, in most published papers of clinical prediction models, the point estimates of the prediction accuracy measures are corrected by adequate bootstrap-based correction methods, but their confidence intervals are not corrected, e.g., the DeLong's confidence interval is usually used for assessing the C-statistic. These naive methods do not adjust for the optimism bias and do not account for statistical variability in the estimation of parameters in the pred…
Bootstrap inference in factor-augmented regressions with weak factors
Center for Data Science, Waseda University（早稻田大学数据科学中心）；Department of Economics and Related Studies, University of York（约克大学经济与相关研究系）；Graduate School of Economics and Management, Tohoku University（东北大学经济管理学院）
Existing frequency domain methods for bootstrapping time series have a limited range. Consider for instance the class of spectral mean statistics (also called integrated periodograms) which includes many important statistics in time series analysis, such as sample autocovariances and autocorrelations among other things. Essentially, such frequency domain bootstrap procedures cover the case of linear time series with independent innovations, and some even require the time series to be Gaussian. In this paper we propose a new, frequency domain bootstrap method which is consistent for a much wide…
Accurate approximation of the sampling distribution of nonparametric kernel density estimators is crucial for many statistical inference problems. Since these estimators have complex asymptotic distributions, bootstrap methods are often used for this purpose. With i.i.d. observations, a large literature exists concerning optimal bootstrap methods which achieve the fastest possible convergence rate of the bootstrap estimator of the sampling distribution of the kernel density estimator. With dependent data, such an optimality theory is an important open problem. We establish a general theory of …
We establish a general theory of optimality for block bootstrap distribution estimation for sample quantiles under a mild strong mixing assumption. In contrast to existing results, we study the block bootstrap for varying numbers of blocks. This corresponds to a hybrid between the subsampling bootstrap and the moving block bootstrap (MBB), in which the number of blocks is somewhere between 1 and the ratio of sample size to block length. Our main theorem determines the optimal choice of the number of blocks and block length to achieve the best possible convergence rate for the block bootstrap d…
This paper presents applications of the peaks-over threshold methodology for both the univariate and the recently introduced bivariate case, combined with a novel bootstrap approach. We compare the proposed bootstrap methods to the more traditional profile likelihood. We have investigated 63 years of the European Climate Assessment daily precipitation data for five Hungarian grid points, first separately for the summer and winter months, then aiming at the detection of possible changes by investigating 20 years moving windows. We show that significant changes can be observed both in the univar…
Mack's distribution-free chain ladder reserving model belongs to the most popular approaches in non-life insurance mathematics. Proposed to determine the first two moments of the reserve, it does not allow to identify the whole distribution of the reserve. For this purpose, Mack's model is usually equipped with a tailor-made bootstrap procedure. Although widely used in practice to estimate the reserve risk, no theoretical bootstrap consistency results exist that justify this approach. To fill this gap in the literature, we adopt the framework proposed by Steinmetz and Jentsch (2022) to derive …
In this paper, we consider possibly misspecified stochastic differential equation models driven by Lévy processes. Regardless of whether the driving noise is Gaussian or not, Gaussian quasi-likelihood estimator can estimate unknown parameters in the drift and scale coefficients. However, in the misspecified case, the asymptotic distribution of the estimator varies by the correction of the misspecification bias, and consistent estimators for the asymptotic variance proposed in the correctly specified case may lose theoretical validity. As one of its solutions, we propose a bootstrap method for …
This paper highlights a tension between semiparametric efficiency and bootstrap consistency in the context of a canonical semiparametric estimation problem, namely the problem of estimating the average density. It is shown that although simple plug-in estimators suffer from bias problems preventing them from achieving semiparametric efficiency under minimal smoothness conditions, the nonparametric bootstrap automatically corrects for this bias and that, as a result, these seemingly inferior estimators achieve bootstrap consistency under minimal smoothness conditions. In contrast, several "debi…
We show that, when the double bootstrap is used to improve performance of bootstrap methods for bias correction, techniques based on using a single double-bootstrap sample for each single-bootstrap sample can be particularly effective. In particular, they produce third-order accuracy for much less computational expense than is required by conventional double-bootstrap methods. However, this improved level of performance is not available for the single double-bootstrap methods that have been suggested to construct confidence intervals or distribution estimators.
Linear mixed-effects models are commonly used to analyze clustered data structures. There are numerous packages to fit these models in R and conduct likelihood-based inference. The implementation of resampling-based procedures for inference are more limited. In this paper, we introduce the lmeresampler package for bootstrapping nested linear mixed-effects models fit via lme4 or nlme. Bootstrap estimation allows for bias correction, adjusted standard errors and confidence intervals for small samples sizes and when distributional assumptions break down. We will also illustrate how bootstrap resa…
Microbial ecology serves as a foundation for a wide range of scientific and biomedical studies. Rapidly-evolving high-throughput sequencing technology enables the comprehensive search for microbial biomarkers using longitudinal experiments. Such experiments consist of repeated biological observations from each subject over time and are essential in accounting for the high between-subject and within-subject variability. Unfortunately, many of the statistical tests based on parametric models rely on correctly specifying temporal dependence structure which is unavailable in most microbiome data. …
In this paper, we establish a high-dimensional CLT for the sample mean of $p$-dimensional spatial data observed over irregularly spaced sampling sites in $\mathbb{R}^d$, allowing the dimension $p$ to be much larger than the sample size $n$. We adopt a stochastic sampling scheme that can generate irregularly spaced sampling sites in a flexible manner and include both pure increasing domain and mixed increasing domain frameworks. To facilitate statistical inference, we develop the spatially dependent wild bootstrap (SDWB) and justify its asymptotic validity in high dimensions by deriving error b…
Bootstrap methods are increasingly accepted as one of the common approaches in constructing confidence intervals in bibliometric studies. Typical bootstrap methods assume that the statistical population is infinite. When the statistical population is finite, a correction needs to be applied in computing the estimated variance of the estimators and thus constructing confidence intervals. We investigate the effect of overlooking the finiteness assumption of the statistical population using a dataset containing all articles in Web of Science (WoS) for Delft University of Technology from 2006 unti…
The construction of confidence regions for parameter vectors is a difficult problem in the nonparametric setting, particularly when the sample size is not large. The bootstrap has shown promise in solving this problem, but empirical evidence often indicates that some bootstrap methods have difficulty in maintaining the correct coverage probability, while other methods may be unstable, often resulting in very large confidence regions. One way to improve the performance of a bootstrap confidence region is to restrict the shape of the region in such a way that the error term of an expansion is as…
We consider the performance of the bootstrap in high-dimensions for the setting of linear regression, where $p<n$ but $p/n$ is not close to zero. We consider ordinary least-squares as well as robust regression methods and adopt a minimalist performance requirement: can the bootstrap give us good confidence intervals for a single coordinate of $β$? (where $β$ is the true regression vector). We show through a mix of numerical and theoretical work that the bootstrap is fraught with problems. Both of the most commonly used methods of bootstrapping for regression -- residual bootstrap and pairs boo…
This paper considers a new bootstrap procedure to estimate the distribution of high-dimensional $\ell_p$-statistics, i.e. the $\ell_p$-norms of the sum of $n$ independent $d$-dimensional random vectors with $d \gg n$ and $p \in [1, \infty]$. We provide a non-asymptotic characterization of the sampling distribution of $\ell_p$-statistics based on Gaussian approximation and show that the bootstrap procedure is consistent in the Kolmogorov-Smirnov distance under mild conditions on the covariance structure of the data. As an application of the general theory we propose a bootstrap hypothesis test …
Multiple imputation has become one of the most popular approaches for handling missing data in statistical analyses. Part of this success is due to Rubin's simple combination rules. These give frequentist valid inferences when the imputation and analysis procedures are so called congenial and the complete data analysis is valid, but otherwise may not. Roughly speaking, congeniality corresponds to whether the imputation and analysis models make different assumptions about the data. In practice imputation and analysis procedures are often not congenial, such that tests may not have the correct s…
Approximately unbiased tests based on bootstrap probabilities are considered for the exponential family of distributions with unknown expectation parameter vector, where the null hypothesis is represented as an arbitrary-shaped region with smooth boundaries. This problem has been discussed previously in Efron and Tibshirani [Ann. Statist. 26 (1998) 1687-1718], and a corrected p-value with second-order asymptotic accuracy is calculated by the two-level bootstrap of Efron, Halloran and Holmes [Proc. Natl. Acad. Sci. U.S.A. 93 (1996) 13429-13434] based on the ABC bias correction of Efron [J. Amer…
Subsampling and block-based bootstrap methods have been used in a wide range of inference problems for time series. To accommodate the dependence, these resampling methods involve a bandwidth parameter, such as subsampling window width and block size in the block-based bootstrap. In empirical work, using different bandwidth parameters could lead to different inference results, but the traditional first order asymptotic theory does not capture the choice of the bandwidth. In this article, we propose to adopt the fixed-b approach, as advocated by Kiefer and Vogelsang (2005) in the heteroscedasti…
Bootstrap smoothed (bagged) parameter estimators have been proposed as an improvement on estimators found after preliminary data-based model selection. The key result of Efron (2014) is a very convenient and widely applicable formula for a delta method approximation to the standard deviation of the bootstrap smoothed estimator. This approximation provides an easily computed guide to the accuracy of this estimator. In addition, Efron (2014) proposed a confidence interval centered on the bootstrap smoothed estimator, with width proportional to the estimate of this approximation to the standard d…
A general approach to selective inference is considered for hypothesis testing of the null hypothesis represented as an arbitrary shaped region in the parameter space of multivariate normal model. This approach is useful for hierarchical clustering where confidence levels of clusters are calculated only for those appeared in the dendrogram, thus subject to heavy selection bias. Our computation is based on a raw confidence measure, called bootstrap probability, which is easily obtained by counting how many times the same cluster appears in bootstrap replicates of the dendrogram. We adjust the b…
Non-negative matrix factorization (NMF) is a technique for finding latent representations of data. The method has been applied to corpora to construct topic models. However, NMF has likelihood assumptions which are often violated by real document corpora. We present a double parametric bootstrap test for evaluating the fit of an NMF-based topic model based on the duality of the KL divergence and Poisson maximum likelihood estimation. The test correctly identifies whether a topic model based on an NMF approach yields reliable results in simulated and real data.
We consider bootstrap-based testing for threshold effects in non-linear threshold autoregressive (TAR) models. It is well-known that classic tests based on asymptotic theory tend to be oversized in the case of small, or even moderate sample sizes, or when the estimated parameters indicate non-stationarity, as often witnessed in the analysis of financial or climate data. To address the issue we propose a supremum Lagrange Multiplier test statistic (sLMb), where the null hypothesis specifies a linear autoregressive (AR) model against the alternative of a TAR model. We consider a recursive bootst…
In distributed, or privacy-preserving learning, we are often given a set of probabilistic models estimated from different local repositories, and asked to combine them into a single model that gives efficient statistical estimation. A simple method is to linearly average the parameters of the local models, which, however, tends to be degenerate or not applicable on non-convex models, or models with different parameter dimensions. One more practical strategy is to generate bootstrap samples from the local models, and then learn a joint model based on the combined bootstrap set. Unfortunately, t…
Consider $M$-estimation in a semiparametric model that is characterized by a Euclidean parameter of interest and an infinite-dimensional nuisance parameter. As a general purpose approach to statistical inferences, the bootstrap has found wide applications in semiparametric $M$-estimation and, because of its simplicity, provides an attractive alternative to the inference approach based on the asymptotic distribution theory. The purpose of this paper is to provide theoretical justifications for the use of bootstrap as a semiparametric inferential tool. We show that, under general conditions, the…
Suppose that $Y = ψ(X_1, \ldots, X_p)$, where $(X_1,\ldots, X_p)^\top$ are random inputs, $Y$ is the output, and $ψ(\cdot)$ is an unknown link function. The Sobol indices gauge the sensitivity of each $X$ against $Y$ by estimating the regression curve's variability between them. In this paper, we estimate these curves with a kernel-based method. The method allows to estimate the first order indices when the link between the independent and dependent variables is unknown. The kernel-based methods need a bandwidth to average the observations. For finite samples, the cross-validation method is fa…
The goal of this paper is to develop a practical and general-purpose approach to construct confidence intervals for differentially private parametric estimation. We find that the parametric bootstrap is a simple and effective solution. It cleanly reasons about variability of both the data sample and the randomized privacy mechanism and applies "out of the box" to a wide class of private estimation routines. It can also help correct bias caused by clipping data to limit sensitivity. We prove that the parametric bootstrap gives consistent confidence intervals in two broadly relevant settings, in…
Upper Confidence Bound (UCB) method is arguably the most celebrated one used in online decision making with partial information feedback. Existing techniques for constructing confidence bounds are typically built upon various concentration inequalities, which thus lead to over-exploration. In this paper, we propose a non-parametric and data-dependent UCB algorithm based on the multiplier bootstrap. To improve its finite sample performance, we further incorporate second-order correction into the above construction. In theory, we derive both problem-dependent and problem-independent regret bound…
Inference about a scalar parameter of interest typically relies on the asymptotic normality of common likelihood pivots, such as the signed likelihood root, the score and Wald statistics. Nevertheless, the resulting inferential procedures are known to perform poorly when the dimension of the nuisance parameter is large relative to the sample size and when the information about the parameters is limited. In many such cases, the use of asymptotic normality of analytical modifications of the signed likelihood root is known to recover inferential performance. It is proved here that parametric boot…
Bootstrap-based estimation and inference for measurement precision under ISO 5725
Research Institute of Science for Safety and Sustainability, National Institute of Advanced Industrial Science and Technology (AIST)（国立产业技术综合研究所安全与可持续性科学研究所）；Department of Industrial and System Engineering, Faculty of Science and Technology, Tokyo University of Science（东京理科大学理工学部工业系统工程系）
We present a new robust bootstrap method for a test when there is a nuisance parameter under the alternative, and some parameters are possibly weakly or non-identified. We focus on a Bierens (1990)-type conditional moment test of omitted nonlinearity for convenience, and because of difficulties that have been ignored to date. Existing methods include the supremum p-value which promotes a conservative test that is generally not consistent, and test statistic transforms like the supremum and average for which bootstrap methods are not valid under weak identification. We propose a new wild bootst…
The bootstrap is a popular and convenient method for quantifying the authority of an empirical ordering of attributes, for example of a ranking of the performance of institutions or of the influence of genes on a response variable. In the first of these examples, the number, $p$, of quantities being ordered is sometimes only moderate in size; in the second it can be very large, often much greater than sample size. However, we show that in both types of problem the conventional bootstrap can produce inconsistency. Moreover, the standard $n$-out-of-$n$ bootstrap estimator of the distribution of …
Universidade Federal de Santa Maria（圣玛丽亚联邦大学）；UFPE（伯南布哥联邦大学）
In this paper, we propose five prediction intervals for the beta autoregressive moving average model. This model is suitable for modeling and forecasting variables that assume values in the interval $(0,1)$. Two of the proposed prediction intervals are based on approximations considering the normal distribution and the quantile function of the beta distribution. We also consider bootstrap-based prediction intervals, namely: (i) bootstrap prediction errors (BPE) interval; (ii) bias-corrected and acceleration (BCa) prediction interval; and (iii) percentile prediction interval based on the quanti…
Selection of important covariates and to drop the unimportant ones from a high-dimensional regression model is a long standing problem and hence have received lots of attention in the last two decades. After selecting the correct model, it is also important to properly estimate the existing parameters corresponding to important covariates. In this spirit, Fan and Li (2001) proposed Oracle property as a desired feature of a variable selection method. Oracle property has two parts; one is the variable selection consistency (VSC) and the other one is the asymptotic normality. Keeping VSC fixed an…
