We present two approaches for next step linear prediction of long memory time series. The first is based on the truncation of the Wiener-Kolmogorov predictor by restricting the observations to the last $k$ terms, which are the only available values in practice. Part of the mean squared prediction error comes from the truncation, and another part comes from the parametric estimation of the parameters of the predictor. By contrast, the second approach is non-parametric. An AR($k$) model is fitted to the long memory time series and we study the error made with this misspecified model.
This paper provides extensions of the work on subsampling by Bertail et al. (2004) for strongly mixing case to weakly dependent case by application of the results of Doukhan and Louhichi (1999). We investigate properties of smooth and rough subsampling estimators for distributions of converging and extreme statistics when the underlying time series is η or λ-weakly dependent.
We prove uniform convergence results for the integrated periodogram of a weakly dependent time series, namely a law of large numbers and a central limit theorem. These results are applied to Whittle's parametric estimation. Under general weak-dependence assumptions we derive uniform limit theorems and asymptotic normality of Whittle's estimate for a large class of models. For instance the causal $θ$-weak dependence property allows a new and unified proof of those results for ARCH($\infty$) and bilinear processes. Non causal $η$-weak dependence yields the same limit theorems for two-sided linea…
In this paper we propose using a nonparametric model specification test for parametric time series with long-range dependence (LRD). To establish asymptotic distributions of the proposed test statistic, we develop new central limit theorems for certain weighted quadratic forms of stationary time series with LRD. To implement our proposed test in practice, we develop a computer-intensive parametric bootstrap simulation procedure for finding simulated critical values. As a result, our finite-sample studies demonstrate that both the proposed theory and the simulation procedure work well, and that…
Driven by network intrusion detection, we propose a MultiResolution Anomaly Detection (MRAD) method, which effectively utilizes the multiscale properties of Internet features and network anomalies. In this paper, several theoretical properties of the MRAD method are explored. A major new result is the mathematical formulation of the notion that a two-scaled MRAD method has larger power than the average power of the detection method based on the given two scales. Test threshold is also developed. Comparisons between MRAD method and other classical outlier detectors in time series are reported a…
The Hurst exponent is the simplest numerical summary of self-similar long-range dependent stochastic processes. We consider the estimation of Hurst exponent in long-range dependent curve time series. Our estimation method begins by constructing an estimate of the long-run covariance function, which we use, via dynamic functional principal component analysis, in estimating the orthonormal functions spanning the dominant sub-space of functional time series. Within the context of functional autoregressive fractionally integrated moving average models, we compare finite-sample bias, variance and m…
Suppose that we observe a short time series where each time-t-specific data-structure consists of many slightly dependent data indexed by a and that we want to estimate a feature of the law of the experiment that depends neither on t nor on a. We develop and study an algorithm to learn sequentially which base algorithm in a user-supplied collection best carries out the estimation task in terms of excess risk and oracular inequalities. The analysis, which uses dependency graph to model the amount of conditional independence within each t-specific data-structure and a concentration inequality by…
We propose a testing procedure based on the Wilcoxon two-sample test statistic in order to test for change-points in the mean of long-range dependent data. We show that the corresponding self-normalized test statistic converges in distribution to a non-degenerate limit under the hypothesis that no change occurred and that it diverges to infinity under the alternative of a change-point with constant height. Furthermore, we derive the asymptotic distribution of the self-normalized Wilcoxon test statistic under local alternatives, that is under the assumption that the height of the level shift de…
Assuming that $(X_t)_{t\in\Z}$ is a vector valued time series with a common marginal distribution admitting a density $f$, our aim is to provide a wide range of consistent estimators of $f$. We consider different methods of estimation of the density as kernel, projection or wavelets ones. Various cases of weakly dependent series are investigated including the Doukhan & Louhichi (1999)'s $η$-weak dependence condition, and the $\tilde ϕ$-dependence of Dedecker & Prieur (2005). We thus obtain results for Markov chains, dynamical systems, bilinear models, non causal Moving Average... From a moment…
Observing a stationary time series, we propose a two-step procedure for the prediction of the next value of the time series. The first step follows machine learning theory paradigm and consists in determining a set of possible predictors as randomized estimators in (possibly numerous) different predictive models. The second step follows the model selection paradigm and consists in choosing one predictor with good properties among all the predictors of the first steps. We study our procedure for two different types of bservations: causal Bernoulli shifts and bounded weakly dependent processes. …
Simultaneous Inference for Covariance and Precision Matrices of Long-Range Dependent Time Series
Booth School of Business, The University of Chicago（芝加哥大学布斯商学院）；Marshall School of Business, University of Southern California（南加州大学马歇尔商学院）；Department of Statistics and Data Science, Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学统计与数据科学系）；Department of Statistics, The University of Chicago（芝加哥大学统计学系）
We analyze the ordinal structure of long-range dependent time series. To this end, we use so called ordinal patterns which describe the relative position of consecutive data points. We provide two estimators for the probabilities of ordinal patterns and prove limit theorems in different settings, namely stationarity and (less restrictive) stationary increments. In the second setting, we encounter a Rosenblatt distribution in the limit. We prove more general limit theorems for functions with Hermite rank 1 and 2. We derive the limit distribution for an estimation of the Hurst parameter $H$ if i…
We consider change-point tests based on rank statistics to test for structural changes in long-range dependent observations. Under the hypothesis of stationary time series and under the assumption of a change with decreasing change-point height, the asymptotic distributions of corresponding test statistics are derived. For this, a uniform reduction principle for the sequential empirical process in a two-parameter Skorohod space equipped with a weighted supremum norm is proved. Moreover, we compare the efficiency of rank tests resulting from the consideration of different score functions. Under…
In this paper we tackle the problem of fast rates in time series forecasting from a statistical learning perspective. In a serie of papers (e.g. Meir 2000, Modha and Masry 1998, Alquier and Wintenberger 2012) it is shown that the main tools used in learning theory with iid observations can be extended to the prediction of time series. The main message of these papers is that, given a family of predictors, we are able to build a new predictor that predicts the series as well as the best predictor in the family, up to a remainder of order $1/\sqrt{n}$. It is known that this rate cannot be improv…
We study the aggregation/disaggregation problem of random parameter AR(1) processes and its relation to the long memory phenomenon. We give a characterization of a subclass of aggregated processes which can be obtained from simpler, "elementary", cases. In particular cases of the mixture densities, the structure (moving average representation) of the aggregated process is investigated.
Long Range Dependence (LRD) in functional sequences is characterized in the spectral domain under suitable conditions. Particularly, multifractionally integrated functional autoregressive moving averages processes can be introduced in this framework. The convergence to zero in the Hilbert-Schmidt operator norm of the integrated bias of the periodogram operator is proved. Under a Gaussian scenario, a weak--consistent parametric estimator of the long--memory operator is then obtained by minimizing, in the norm of bounded linear operators, a divergence information functional loss.
We consider a change-point test based on the Hill estimator to test for structural changes in the tail index of Long Memory Stochastic Volatility time series. In order to determine the asymptotic distribution of the corresponding test statistic, we prove a uniform reduction principle for the tail empirical process in a two-parameter Skorohod space. It is shown that such a process displays a dichotomous behavior according to an interplay between the Hurst parameter, i.e., a parameter characterizing the dependence in the data, and the tail index. Our theoretical results are accompanied by simula…
This article develops a periodic version of a time varying parameter fractional process in the stationary region. It is a partial extension of Hosking (1981)'s article which dealt with the case where the coefficients are invariant in time. We will describe the probabilistic theories of this periodic model. The results are followed by a graphical representation of the autocovariances functions.
This article deals with detection of nonconstant long memory parameter in time series. The null hypothesis presumes stationary or nonstationary time series with constant long memory parameter, typically an I(d) series with d>-.5. The alternative corresponds to an increase in persistence and includes in particular an abrupt or gradual change from I(d_1) to I(d_2).
We discuss existence and uniqueness of stationary and ergodic nonlinear autoregressive processes when exogenous regressors are incorporated in the dynamic. To this end, we consider the convergence of the backward iterations of dependent random maps. In particular, we give a new result when the classical condition of contraction on average is replaced with a contraction in conditional expectation. Under some conditions, we also derive an explicit control of the functional dependence of Wu (2005) which guarantees a wide range of statistical applications. Our results are illustrated with CHARN mo…
In this paper, we develop new multiscale methods to test qualitative hypotheses about the regression function m in a nonparametric regression model with fixed design points and time series errors. In time series applications, m represents a nonparametric time trend. Practitioners are often interested in whether the trend m has certain shape properties. For example, they would like to know whether m is constant or whether it is increasing/decreasing in certain time regions. Our multiscale methods allow to test for such shape properties of the trend m. In order to perform the methods, we require…
In this paper, we obtain general representations for the joint distributions and copulas of arbitrary dependent random variables absolutely continuous with respect to the product of given one-dimensional marginal distributions. The characterizations obtained in the paper represent joint distributions of dependent random variables and their copulas as sums of $U$-statistics in independent random variables. We show that similar results also hold for expectations of arbitrary statistics in dependent random variables. As a corollary of the results, we obtain new representations for multivariate di…
We find the asymptotic distribution of the sample autocovariances of long-memory processes in cases of finite and infinite fourth moment. Depending on the interplay of assumptions on moments and the intensity of dependence, there are three types of convergence rates and limit distributions. In particular, a normal approximation with the standard rate does not always hold in practically relevant cases.
This paper shows how a time series of measurements of an evolving system can be processed to create an inner time series that is unaffected by any instantaneous invertible, possibly nonlinear transformation of the measurements. An inner time series contains information that does not depend on the nature of the sensors, which the observer chose to monitor the system. Instead, it encodes information that is intrinsic to the evolution of the observed system. Because of its sensor-independence, an inner time series may produce fewer false negatives when it is used to detect events in the presence …
Kohonen self-organisation maps are a well know classification tool, commonly used in a wide variety of problems, but with limited applications in time series forecasting context. In this paper, we propose a forecasting method specifically designed for multi-dimensional long-term trends prediction, with a double application of the Kohonen algorithm. Practical applications of the method are also presented.
Expectiles define the only law-invariant, coherent and elicitable risk measure apart from the expectation. The popularity of expectile-based risk measures is steadily growing and their properties have been studied for independent data, but further results are needed to use extreme expectiles with dependent time series such as financial data. In this paper we establish a basis for inference on extreme expectiles and expectile-based marginal expected shortfall in a general $β$-mixing context that encompasses ARMA, ARCH and GARCH models with heavy-tailed innovations. Simulations and applications …
The problem of change-point estimation is considered under a general framework where the data are generated by unknown stationary ergodic process distributions. In this context, the consistent estimation of the number of change-points is provably impossible. However, it is shown that a consistent clustering method may be used to estimate the number of change points, under the additional constraint that the correct number of process distributions that generate the data is provided. This additional parameter has a natural interpretation in many real-world applications. An algorithm is proposed t…
For the challenging task of modeling multivariate time series, we propose a new class of models that use dependent Matérn processes to capture the underlying structure of data, explain their interdependencies, and predict their unknown values. Although similar models have been proposed in the econometric, statistics, and machine learning literature, our approach has several advantages that distinguish it from existing methods: 1) it is flexible to provide high prediction accuracy, yet its complexity is controlled to avoid overfitting; 2) its interpretability separates it from black-box methods…
For temporal regularly spaced datasets, a lot of methods are available and the properties of these methods are extensively investigated. Less research has been performed on irregular temporal datasets subject to measurement error with complex dependence structures, while this type of datasets is widely available. In this paper, the performance of kernel smoother for trend and its derivatives is considered under dependent measurement errors and irregularly spaced sampling scheme. The error processes are assumed to be subordinated Gaussian long memory processes and have varying marginal distribu…
Inferring causal relations from observational time series data is a key problem across science and engineering whenever experimental interventions are infeasible or unethical. Increasing data availability over the past decades has spurred the development of a plethora of causal discovery methods, each addressing particular challenges of this difficult task. In this paper we focus on an important challenge that is at the core of time series causal discovery: regime-dependent causal relations. Often dynamical systems feature transitions depending on some, often persistent, unobserved background …
A dynamic Boltzmann machine (DyBM) has been proposed as a model of a spiking neural network, and its learning rule of maximizing the log-likelihood of given time-series has been shown to exhibit key properties of spike-timing dependent plasticity (STDP), which had been postulated and experimentally confirmed in the field of neuroscience as a learning rule that refines the Hebbian rule. Here, we relax some of the constraints in the DyBM in a way that it becomes more suitable for computation and learning. We show that learning the DyBM can be considered as logistic regression for binary-valued t…
This paper presents a bootstrapped p-value white noise test based on the maximum correlation, for a time series that may be weakly dependent under the null hypothesis. The time series may be prefiltered residuals. The test statistic is a normalized weighted maximum sample correlation, where the maximum lag increases at a rate slower than the sample size. We only require uncorrelatedness under the null hypothesis, along with a moment contraction dependence property that includes mixing and non-mixing sequences. We show Shao's (2011) dependent wild bootstrap is valid for a much larger class of p…
We consider the problem of inferring the conditional independence graph (CIG) of a high-dimensional stationary multivariate Gaussian time series. In a time series graph, each component of the vector series is represented by distinct node, and associations between components are represented by edges between the corresponding nodes. We formulate the problem as one of multi-attribute graph estimation for random vectors where a vector is associated with each node of the graph. At each node, the associated random vector consists of a time series component and its delayed copies. We present an alter…
The following working document summarizes our work on the clustering of financial time series. It was written for a workshop on information geometry and its application for image and signal processing. This workshop brought several experts in pure and applied mathematics together with applied researchers from medical imaging, radar signal processing and finance. The authors belong to the latter group. This document was written as a long introduction to further development of geometric tools in financial applications such as risk or portfolio analysis. Indeed, risk and portfolio analysis essent…
University of Washington（华盛顿大学）；TCS Research（塔塔咨询服务公司研究院）
Systematic generalization aims to evaluate reasoning about novel combinations from known components, an intrinsic property of human cognition. In this work, we study systematic generalization of NNs in forecasting future time series of dependent variables in a dynamical system, conditioned on past time series of dependent variables, and past and future control variables. We focus on systematic generalization wherein the NN-based forecasting model should perform well on previously unseen combinations or regimes of control variables after being trained on a limited set of the possible regimes. F…
In this work we propose a new class of long-memory models with time-varying fractional parameter. In particular, the dynamics of the long-memory coefficient, $d$, is specified through a stochastic recurrence equation driven by the score of the predictive likelihood, as suggested by Creal et al. (2013) and Harvey (2013). We demonstrate the validity of the proposed model by a Monte Carlo experiment and an application to two real time series.
University of Sherbrooke（舍布鲁克大学）；University of Quebec at Montreal（魁北克大学蒙特利尔分校）
We investigate the problem of discovering and modeling regime shifts in an ecosystem comprising multiple time series known as co-evolving time series. Regime shifts refer to the changing behaviors exhibited by series at different time intervals. Learning these changing behaviors is a key step toward time series forecasting. While advances have been made, existing methods suffer from one or more of the following shortcomings: (1) failure to take relationships between time series into consideration for discovering regimes in multiple time series; (2) lack of an effective approach that models tim…
Researchers have used from 30 days to several years of daily returns as source data for clustering financial time series based on their correlations. This paper sets up a statistical framework to study the validity of such practices. We first show that clustering correlated random variables from their observed values is statistically consistent. Then, we also give a first empirical answer to the much debated question: How long should the time series be? If too short, the clusters found can be spurious; if too long, dynamics can be smoothed out.
This article proposes a new approach to modeling high-dimensional time series by treating a $p$-dimensional time series as a nonsingular linear transformation of certain common factors and idiosyncratic components. Unlike the approximate factor models, we assume that the factors capture all the non-trivial dynamics of the data, but the cross-sectional dependence may be explained by both the factors and the idiosyncratic components. Under the proposed model, (a) the factor process is dynamically dependent and the idiosyncratic component is a white noise process, and (b) the largest eigenvalues …
The goodness of the long-term prediction in the state-space model was evaluated using the squared long-term prediction error. In order to estimate the model parameters suitable for long-term prediction, we devised a modified log-likelihood corresponding to the long-term prediction error variance. Trend models and seasonally adjusted models with and without AR component are examined as examples.
This paper studies the daily connectivity time series of a wind speed-monitoring network using multifractal detrended fluctuation analysis. It investigates the long-range fluctuation and multifractality in the residuals of the connectivity time series. Our findings reveal that the daily connectivity of the correlation-based network is persistent for any correlation threshold. Further, the multifractality degree is higher for larger absolute values of the correlation threshold
Uppsala University（乌普萨拉大学）；Nanjing University of Posts and Telecommunications（南京邮电大学）
We consider the problem of learning a classifier from observed functional data. Here, each data-point takes the form of a single time-series and contains numerous features. Assuming that each such series comes with a binary label, the problem of learning to predict the label of a new coming time-series is considered. Hereto, the notion of {\em margin} underlying the classical support vector machine is extended to the continuous version for such data. The longitudinal support vector machine is also a convex optimization problem and its dual form is derived as well. Empirical results for specifi…
Long Short-Term Memory (LSTM) networks are often used to capture temporal dependency patterns. By stacking multi-layer LSTM networks, it can capture even more complex patterns. This paper explores the effectiveness of applying stacked LSTM networks in the time series prediction domain, specifically, the traffic volume forecasting. Being able to predict traffic volume more accurately can result in better planning, thus greatly reduce the operation cost and improve overall efficiency.
