A bootstrap procedure for constructing prediction bands for a stationary functional time series is proposed. The procedure exploits a general vector autoregressive representation of the time-reversed series of Fourier coefficients appearing in the Karhunen-Loeve representation of the functional process. It generates backward-in-time, functional replicates that adequately mimic the dependence structure of the underlying process in a model-free way and have the same conditionally fixed curves at the end of each functional pseudo-time series. The bootstrap prediction error distribution is then calculated as the difference between the model-free, bootstrap-generated future functional observations and the functional forecasts obtained from the model used for prediction. This allows the estimated prediction error distribution to account for the innovation and estimation errors associated with prediction and the possible errors due to model misspecification. We establish the asymptotic validity of the bootstrap procedure in estimating the conditional prediction error distribution of interest, and we also show that the procedure enables the construction of prediction bands that achieve (asymptotically) the desired coverage. Prediction bands based on a consistent estimation of the conditional distribution of the studentized prediction error process also are introduced. Such bands allow for taking more appropriately into account the local uncertainty of prediction. Through a simulation study and the analysis of two data sets, we demonstrate the capabilities and the good finite-sample performance of the proposed method.
A model-free bootstrap procedure for a general class of stationary time series is introduced. The theoretical framework is established, showing asymptotic validity of bootstrap confidence intervals for many statistics of interest. In addition, asymptotic validity of one-step ahead bootstrap prediction intervals is also demonstrated. Finite-sample experiments are conducted to empirically confirm the performance of the new method, and to compare with popular methods such as the block bootstrap and the autoregressive (AR)-sieve bootstrap.
A bootstrap procedure for functional time series is proposed which exploits a general vector autoregressive representation of the time series of Fourier coefficients appearing in the Karhunen-Loève expansion of the functional process. A double sieve-type bootstrap method is developed which avoids the estimation of process operators and generates functional pseudo-time series that appropriately mimic the dependence structure of the functional time series at hand. The method uses a finite set of functional principal components to capture the essential driving parts of the infinite dimensional pr…
Functional times series have become an integral part of both functional data and time series analysis. This paper deals with the functional autoregressive model of order 1 and the autoregression bootstrap for smooth functions. The regression operator is estimated in the framework developed by Ferraty and Vieu [2004] and Ferraty et al. [2007] which is here extended to the double functional case under an assumption of stationary ergodic data which dates back to Laib and Louani [2010]. The main result of this article is the characterization of the asymptotic consistency of the bootstrapped regres…
The second-order dependence structure of purely nondeterministic stationary process is described by the coefficients of the famous Wold representation. These coefficients can be obtained by factorizing the spectral density of the process. This relation together with some spectral density estimator is used in order to obtain consistent estimators of these coefficients. A spectral density-driven bootstrap for time series is then developed which uses the entire sequence of estimated MA coefficients together with appropriately generated pseudo innovations in order to obtain a bootstrap pseudo time…
We consider infinite-dimensional Hilbert space-valued random variables that are assumed to be temporal dependent in a broad sense. We prove a central limit theorem for the moving block bootstrap and for the tapered block bootstrap, and show that these block bootstrap procedures also provide consistent estimators of the long run covariance operator. Furthermore, we consider block bootstrap-based procedures for fully functional testing of the equality of mean functions between several independent functional time series. We establish validity of the block bootstrap methods in approximating the di…
A new time series bootstrap scheme, the time frequency toggle (TFT)-bootstrap, is proposed. Its basic idea is to bootstrap the Fourier coefficients of the observed time series, and then to back-transform them to obtain a bootstrap sample in the time domain. Related previous proposals, such as the "surrogate data" approach, resampled only the phase of the Fourier coefficients and thus had only limited validity. By contrast, we show that the appropriate resampling of phase and magnitude, in addition to some smoothing of Fourier coefficients, yields a bootstrap scheme that mimics the correct seco…
For testing hypothesis on the covariance operator of functional time series, we suggest to use the full functional information and to avoid dimension reduction techniques. The limit distribution follows from the central limit theorem of the weak convergence of the partial sum process in general Hilbert space applied to the product space. In order to obtain critical values for tests, we generalize bootstrap results from the independent to the dependent case. This results can be applied to covariance operators, autocovariance operators and cross covariance operators. We discuss one sample and ch…
This article studies bootstrap inference for high dimensional weakly dependent time series in a general framework of approximately linear statistics. The following high dimensional applications are covered: (1) uniform confidence band for mean vector; (2) specification testing on the second order property of time series such as white noise testing and bandedness testing of covariance matrix; (3) specification testing on the spectral property of time series. In theory, we first derive a Gaussian approximation result for the maximum of a sum of weakly dependent vectors, where the dimension of th…
We consider the problem of finding confidence intervals for the risk of forecasting the future of a stationary, ergodic stochastic process, using a model estimated from the past of the process. We show that a bootstrap procedure provides valid confidence intervals for the risk, when the data source is sufficiently mixing, and the loss function and the estimator are suitably smooth. Autoregressive (AR(d)) models estimated by least squares obey the necessary regularity conditions, even when mis-specified, and simulations show that the finite- sample coverage of our bounds quickly converges to th…
The aim of this paper it to establish sufficient conditions for consistency of moving block bootstrap for non-stationary time series with periodic and almost periodic structure. The parameter of the study is the mean value of the expectation function. Consistency holds in quite general situations: if all joint distributions of the series are periodic, then it suffices to assume the central limit theorem and strong mixing property, together with summability of the autocovariance function. In the case where the mean function is almost periodic, we additionally need uniform boundedness of the fou…
Multivariate time series present many challenges, especially when they are high dimensional. The paper's focus is twofold. First, we address the subject of consistently estimating the autocovariance sequence; this is a sequence of matrices that we conveniently stack into one huge matrix. We are then able to show consistency of an estimator based on the so-called flat-top tapers; most importantly, the consistency holds true even when the time series dimension is allowed to increase with the sample size. Second, we revisit the linear process bootstrap (LPB) procedure proposed by McMurry and Poli…
We consider strictly stationary stochastic processes of Hilbert space-valued random variables and focus on fully functional tests for the equality of the lag-zero autocovariance operators of several independent functional time series. A moving block bootstrap-based testing procedure is proposed which generates pseudo random elements that satisfy the null hypothesis of interest. It is based on directly bootstrapping the time series of tensor products which overcomes some common difficulties associated with applications of the bootstrap to related testing problems. The suggested methodology can …
The problem of comparing the entire second order structure of two functional processes is considered and a $L^2$-type statistic for testing equality of the corresponding spectral density operators is investigated. The test statistic evaluates, over all frequencies, the Hilbert-Schmidt distance between the two estimated spectral density operators. Under certain assumptions, the limiting distribution under the null hypothesis is derived. A novel frequency domain bootstrap method is introduced, which leads to a more accurate approximation of the distribution of the test statistic under the null t…
For many relevant statistics of multivariate time series, no valid frequency domain bootstrap procedures exist. This is mainly due to the fact that the distribution of such statistics depends on the fourth-order moment structure of the underlying process in nearly every scenario, except for some special cases like Gaussian time series. In contrast to the univariate case, even additional structural assumptions such as linearity of the multivariate process or a standardization of the statistic of interest do not solve the problem. This paper focuses on integrated periodogram statistics as well a…
Bootstrap methods for estimating the long-run covariance of stationary functional time series are considered. We introduce a versatile bootstrap method that relies on functional principal component analysis, where principal component scores can be bootstrapped by maximum entropy. Two other bootstrap methods resample error functions, after the dependence structure being modeled linearly by a sieve method or nonlinearly by a functional kernel regression. Through a series of Monte-Carlo simulation, we evaluate and compare the finite-sample performances of these three bootstrap methods for estimat…
This project revolves around studying estimators for parameters in different Time Series models and studying their assymptotic properties. We introduce various bootstrap techniques for the estimators obtained. Our special emphasis is on Weighted Bootstrap. We establish the consistency of this scheme in a AR model and its variations. Numerical calculations lend further support to our consistency results. Next we analyze ARCH models, and study various estimators used for different error distributions. We also present resampling techniques for estimating the distribution of the estimators. Finall…
The problem of testing equality of the entire second order structure of two independent functional linear processes is considered. A fully functional $L^2$-type test is developed which evaluates, over all frequencies, the Hilbert-Schmidt distance between the estimated spectral density operators of the two processes. The asymptotic behavior of the test statistic is investigated and its limiting distribution under the null hypothesis is derived. Furthermore, a novel frequency domain bootstrap method is developed which approximates more accurately the distribution of the test statistic under the …
Fitting sparse models to high-dimensional time series is an important area of statistical inference. In this paper we consider sparse vector autoregressive models and develop appropriate bootstrap methods to infer properties of such processes. Our bootstrap methodology generates pseudo time series using a model-based bootstrap procedure which involves an estimated, sparsified version of the underlying vector autoregressive model. Inference is performed using so-called de-sparsified or de-biased estimators of the autoregressive model parameters. We derive the asymptotic distribution of such est…
A new portmanteau test statistic is proposed for detecting nonlinearity in time series data. In this paper, we elaborate on the Toeplitz autocorrelation matrix to the autocorrelation and cross-correlation of residuals and squared residuals block matrix. We derive a new portmanteau test statistic using the log of the determinant of the mth autocorrelations and cross-correlations block matrix. The asymptotic distribution of the proposed test statistic is derived as a linear combination of chi-squared distributions and can be approximated by a gamma distribution. This test is applied to identify …
The bootstrap procedure has emerged as a general framework to construct prediction intervals for future observations in autoregressive time series models. Such models with outlying data points are standard in real data applications, especially in the field of econometrics. These outlying data points tend to produce high forecast errors, which reduce the forecasting performances of the existing bootstrap prediction intervals calculated based on non-robust estimators. In the univariate and multivariate autoregressive time series, we propose a robust bootstrap algorithm for constructing predictio…
In Das and Politis(2020), a model-free bootstrap(MFB) paradigm was proposed for generating prediction intervals of univariate, (locally) stationary time series. Theoretical guarantees for this algorithm was resolved in Wang and Politis(2019) under stationarity and weak dependence condition. Following this line of work, here we extend MFB for predictive inference under a multivariate time series setup. We describe two algorithms, the first one works for a particular class of time series under any fixed dimension d; the second one works for a more generalized class of time series under low-dimen…
In this paper, we derive higher order Edgeworth expansions for the finite sample distributions of the subsampling-based t-statistic and the Wald statistic in the Gaussian location model under the so-called fixed-smoothing paradigm. In particular, we show that the error of asymptotic approximation is at the order of the reciprocal of the sample size and obtain explicit forms for the leading error terms in the expansions. The results are used to justify the second-order correctness of a new bootstrap method, the Gaussian dependent bootstrap, in the context of Gaussian location model.
The limiting distribution for M-estimates in a non-stationary autoregressive model with heavy-tailed error is computationally intractable. To make inferences based on the M-estimates, the bootstrap procedure can be used to approximate the sampling distribution. In this paper, we show that the bootstrap scheme with $m=o(n)$ resampling sample size when $m/n \to 0$ is approximately valid in a multiple unit roots time series with innovations in the domain of attraction of a stable law with index $0<α\leq2$.
Davis and Mikosch [7] introduced the extremogram as a flexible quantitative tool for measuring various types of extremal dependence in a stationary time series. There we showed some standard statistical properties of the sample extremogram. A major difficulty was the construction of credible confidence bands for the extremogram. In this paper, we employ the stationary bootstrap to overcome this problem. Moreover, we introduce the cross extremogram as a measure of extremal serial dependence between two or more time series. We also study the extremogram for return times between extremal events. …
University of Southern Denmark（南丹麦大学）；University of Bristol（布里斯托大学）
We propose a novel bootstrap procedure for dependent data based on Generative Adversarial networks (GANs). We show that the dynamics of common stationary time series processes can be learned by GANs and demonstrate that GANs trained on a single sample path can be used to generate additional samples from the process. We find that temporal convolutional neural networks provide a suitable design for the generator and discriminator, and that convincing samples can be generated on the basis of a vector of iid normal noise. We demonstrate the finite sample properties of GAN sampling and the suggeste…
A time-domain test for the assumption of second order stationarity of a functional time series is proposed. The test is based on combining individual cumulative sum tests which are designed to be sensitive to changes in the mean, variance and autocovariance operators, respectively. The combination of their dependent $p$-values relies on a joint dependent block multiplier bootstrap of the individual test statistics. Conditions under which the proposed combined testing procedure is asymptotically valid under stationarity are provided. A procedure is proposed to automatically choose the block len…
To draw inference on serial extremal dependence within heavy-tailed Markov chains, Drees, Segers and Warchoł [Extremes (2015) 18, 369--402] proposed nonparametric estimators of the spectral tail process. The methodology can be extended to the more general setting of a stationary, regularly varying time series. The large-sample distribution of the estimators is derived via empirical process theory for cluster functionals. The finite-sample performance of these estimators is evaluated via Monte Carlo simulations. Moreover, two different bootstrap schemes are employed which yield confidence inter…
Solving real-world problems, particularly with deep learning, relies on the availability of abundant, quality data. In this paper we develop a novel framework that maximises the utility of time-series datasets that contain only small quantities of expertly-labelled data, larger quantities of weakly (or coarsely) labelled data and a large volume of unlabelled data. This represents scenarios commonly encountered in the real world, such as in crowd-sourcing applications. In our work, we use a nested loop using a Kernel Density Estimator (KDE) to super-resolve the abundant low-quality data labels,…
This paper presents a bootstrapped p-value white noise test based on the maximum correlation, for a time series that may be weakly dependent under the null hypothesis. The time series may be prefiltered residuals. The test statistic is a normalized weighted maximum sample correlation, where the maximum lag increases at a rate slower than the sample size. We only require uncorrelatedness under the null hypothesis, along with a moment contraction dependence property that includes mixing and non-mixing sequences. We show Shao's (2011) dependent wild bootstrap is valid for a much larger class of p…
This paper proposes the cross-quantilogram to measure the quantile dependence between two time series. We apply it to test the hypothesis that one time series has no directional predictability to another time series. We establish the asymptotic distribution of the cross quantilogram and the corresponding test statistic. The limiting distributions depend on nuisance parameters. To construct consistent confidence intervals we employ the stationary bootstrap procedure; we show the consistency of this bootstrap. Also, we consider the self-normalized approach, which is shown to be asymptotically pi…
The functional delta-method provides a convenient tool for deriving the asymptotic distribution of a plug-in estimator of a statistical functional from the asymptotic distribution of the respective empirical process. Moreover, it provides a tool to derive bootstrap consistency for plug-in estimators from bootstrap consistency of empirical processes. It has recently been shown that the range of applications of the functional delta-method for the asymptotic distribution can be considerably enlarged by employing the notion of quasi-Hadamard differentiability. Here we show in a general setting tha…
The functional delta-method provides a convenient tool for deriving bootstrap consistency of a sequence of plug-in estimators w.r.t. a given functional from bootstrap consistency of the underlying sequence of estimators. It has recently been shown in Beutner and Zähle (2016) that the range of applications of the functional delta-method for establishing bootstrap consistency in probability of the sequence of plug-in estimators can be considerably enlarged by replacing the usual condition of Hadamard differentiability of the given functional by the weaker condition of quasi-Hadamard differentiab…
The bootstrap is a popular and powerful method for assessing precision of estimators and inferential methods. However, for massive datasets which are increasingly prevalent, the bootstrap becomes prohibitively costly in computation and its feasibility is questionable even with modern parallel computing platforms. Recently Kleiner, Talwalkar, Sarkar, and Jordan (2014) proposed a method called BLB (Bag of Little Bootstraps) for massive data which is more computationally scalable with little sacrifice of statistical accuracy. Building on BLB and the idea of fast double bootstrap, we propose a new…
We provide a functional central limit theorem for a broad class of smooth functions for possibly noncausal multivariate linear processes with time-varying coefficients. Since the limiting processes depend on unknown quantities, we propose a local block bootstrap procedure to circumvent this inconvenience in practical applications. In particular, we prove bootstrap validity for a very broad class of processes. Our results are illustrated by some numerical examples.
We consider the residual-based or naive bootstrap for functional autoregressions of order 1 and prove that it is asymptotically valid for, e.g., the sample mean and for empirical covariance operator estimates. As a crucial auxiliary result, we also show that the empirical distribution of the centered sample innovations converges to the distribution of the innovations with respect to the Mallows metric.
A weighted regression procedure is proposed for regression type problems where the innovations are heavy-tailed. This method approximates the least absolute regression method in large samples, and the main advantage will be if the sample is large and for problems with many independent variables. In such problems bootstrap methods must often be utilized to test hypotheses and especially in such a case this procedure has an advantage over least absolute regression. The procedure will be illustrated on first-order autoregressive problems, including the random walk. A bootstrap procedure is used t…
We study an AMOC time series model with an abrupt change in the mean and dependent errors that fulfill certain mixing conditions. We obtain confidence intervals for the unknown change-point via bootstrapping methods. Precisely we use a block bootstrap of the estimated centered error sequence. Then we reconstruct a sequence with a change in the mean using the same estimators as before. The difference between the change-point estimator of the resampled sequence and the one for the original sequence can be use as an approximation of the difference between the real change-point and its estimator. …
The bootstrap provides a simple and powerful means of assessing the quality of estimators. However, in settings involving large datasets---which are increasingly prevalent---the computation of bootstrap-based quantities can be prohibitively demanding computationally. While variants such as subsampling and the $m$ out of $n$ bootstrap can be used in principle to reduce the cost of bootstrap computations, we find that these methods are generally not robust to specification of hyperparameters (such as the number of subsampled data points), and they often require use of more prior information (suc…
Strict stationarity is a common assumption used in the time series literature in order to derive asymptotic distributional results for second-order statistics, like sample autocovariances and sample autocorrelations. Focusing on weak stationarity, this paper derives the asymptotic distribution of the maximum of sample autocovariances and sample autocorrelations under weak conditions by using Gaussian approximation techniques. The asymptotic theory for parameter estimation obtained by fitting a (linear) autoregressive model to a general weakly stationary time series is revisited and a Gaussian …
