Entropy estimation, due in part to its connection with mutual information, has seen considerable use in the study of time series data including causality detection and information flow. In many cases, the entropy is estimated using $k$-nearest neighbor (Kozachenko-Leonenko) based methods. However, analytic results on this estimator are limited to independent data. In the article, we show rigorous bounds on the rate of decay of the bias in the number of samples, $N$, assuming they are drawn from a stationary process which satisfies a suitable mixing condition. Numerical examples are presented which demonstrate the efficiency of the estimator when applied to a Markov process with stationary Gaussian density. These results support the asymptotic rates derived in the theoretical work.
Given a heterogeneous time-series sample, the objective is to find points in time (called change points) where the probability distribution generating the data has changed. The data are assumed to have been generated by arbitrary unknown stationary ergodic distributions. No modelling, independence or mixing assumptions are made. A novel, computationally efficient, nonparametric method is proposed, and is shown to be asymptotically consistent in this general framework. The theoretical results are complemented with experimental evaluations.
Long memory or long range dependency is an important phenomenon that may arise in the analysis of time series or spatial data. Most of the definitions of long memory of a stationary process $X=\{X_1, X_2,\cdots,\}$ are based on the second-order properties of the process. The excess entropy of a stationary process is the summation of redundancies which relates to the rate of convergence of the conditional entropy $H(X_n|X_{n-1},\cdots, X_1)$ to the entropy rate. It is proved that the excess entropy is identical to the mutual information between the past and the future when the entropy $H(X_1)$ …
This chapter presents some novel information theoretic results for the analysis of stationary time series in the frequency domain. In particular, the spectral distribution that corresponds to the most uncertain or unpredictable time series with some values of the autocovariance function fixed, is the generalized von Mises spectral distribution. It is thus a maximum entropy spectral distribution and the corresponding stationary time series is called the generalized von Mises time series. The generalized von Mises distribution is used in directional statistics for modelling planar directions tha…
A model-free bootstrap procedure for a general class of stationary time series is introduced. The theoretical framework is established, showing asymptotic validity of bootstrap confidence intervals for many statistics of interest. In addition, asymptotic validity of one-step ahead bootstrap prediction intervals is also demonstrated. Finite-sample experiments are conducted to empirically confirm the performance of the new method, and to compare with popular methods such as the block bootstrap and the autoregressive (AR)-sieve bootstrap.
Our aim in this work is to give explicit formula of the linear processes solution of autoregressive time series AR(2) with hint of generating functions theory by using the Horadam numbers and polynomials.
The Portmanteau test provides the vanilla method for detecting serial correlations in classical univariate time series analysis. The method is extended to the case of observations from a locally stationary functional time series. Asymptotic critical values are obtained by a suitable block multiplier bootstrap procedure. The test is shown to asymptotically hold its level and to be consistent against general alternatives.
We consider a time series $X=\{X_k, k\in\mathbb{Z}\}$ with memory parameter $d\in\mathbb{R}$. This time series is either stationary or can be made stationary after differencing a finite number of times. We study the "Local Whittle Wavelet Estimator" of the memory parameter $d$. This is a wavelet-based semiparametric pseudo-likelihood maximum method estimator. The estimator may depend on a given finite range of scales or on a range which becomes infinite with the sample size. We show that the estimator is consistent and rate optimal if $X$ is a linear process and is asymptotically normal if $X$…
We propose generalized portmanteau-type test statistics in the frequency domain to test independence between two stationary time series. The test statistics are formed analogous to the one in Chen and Deo (2004, Econometric Theory 20, 382-416), who extended the applicability of portmanteau goodness-of-fit test to the long memory case. Under the null hypothesis of independence, the asymptotic standard normal distributions of the proposed statistics are derived under fairly mild conditions. In particular, each time series is allowed to possess short memory, long memory or anti-persistence. A sim…
Various nonparametric approaches for Bayesian spectral density estimation of stationary time series have been suggested in the literature, mostly based on the Whittle likelihood approximation. A generalization of this approximation has been proposed in Kirch et al. who prove posterior consistency for spectral density estimation in combination with the Bernstein-Dirichlet process prior for Gaussian time series. In this paper, we will extend the posterior consistency result to non-Gaussian time series by employing a general consistency theorem of Shalizi for dependent data and misspecified model…
A measure of primal importance for capturing the serial dependence of a stationary time series at extreme levels is provided by the limiting cluster size distribution. New estimators based on a blocks declustering scheme are proposed and analyzed both theoretically and by means of a large-scale simulation study. A sliding blocks version of the estimators is shown to outperform a disjoint blocks version. In contrast to some competitors from the literature, the estimators only depend on one unknown parameter to be chosen by the statistician.
We study distributional properties of a quadratic form of a stationary functional time series under mild moment conditions. As an important application, we obtain consistency rates of estimators of spectral density operators and prove joint weak convergence to a vector of complex Gaussian random operators. Weak convergence is established based on an approximation of the form via transforms of Hilbert-valued martingale difference sequences. As a side-result, the distributional properties of the long-run covariance operator are established.
We develop an estimator for the high-dimensional covariance matrix of a locally stationary process with a smoothly varying trend and use this statistic to derive consistent predictors in non-stationary time series. In contrast to the currently available methods for this problem the predictor developed here does not rely on fitting an autoregressive model and does not require a vanishing trend. The finite sample properties of the new methodology are illustrated by means of a simulation study and a financial indices study.
The extremal index $θ$, a number in the interval $[0,1]$, is known to be a measure of primal importance for analyzing the extremes of a stationary time series. New rank-based estimators for $θ$ are proposed which rely on the construction of approximate samples from the exponential distribution with parameter $θ$ that is then to be fitted via the method of moments. The new estimators are analyzed both theoretically as well as empirically through a large-scale simulation study. In specific scenarios, in particular for time series models with $θ\approx 1$, they are found to be superior to recent …
Forecasting the evolution of complex systems is one of the grand challenges of modern data science. The fundamental difficulty lies in understanding the structure of the observed stochastic process. In this paper, we show that every uniformly-positive-definite-in-covariance and sufficiently short-range dependent non-stationary and nonlinear time series can be well approximated globally by an auto-regressive process of slowly diverging order. When linear prediction with ${\cal L}^2$ loss is concerned, the latter result facilitates a unified globally-optimal short-term forecasting theory for a w…
In this paper, we consider the estimation and inference of precision matrices of a rich class of locally stationary and nonlinear time series assuming that only one realization of the time series is observed. Using a Cholesky decomposition technique, we show that the precision matrices can be directly estimated via a series of least squares linear regressions with smoothly time-varying coefficients. The method of sieves is utilized for the estimation and is shown to be efficient and optimally adaptive in terms of estimation accuracy and computational complexity. We establish an asymptotic theo…
A time-varying empirical spectral process indexed by classes of functions is defined for locally stationary time series. We derive weak convergence in a function space, and prove a maximal exponential inequality and a Glivenko--Cantelli-type convergence result. The results use conditions based on the metric entropy of the index class. In contrast to related earlier work, no Gaussian assumption is made. As applications, quasi-likelihood estimation, goodness-of-fit testing and inference under model misspecification are discussed. In an extended application, uniform rates of convergence are deriv…
In this paper, we focus on isotropic and stationary sphere-cross-time random fields. We first introduce the class of spherical functional autoregressive-moving average processes (SPHARMA), which extend in a natural way the spherical functional autoregressions (SPHAR) recently studied in [8, 7]; more importantly, we then show that SPHAR and SPHARMA processes of sufficiently large order can be exploited to approximate every isotropic and stationary sphere-cross-time random field, thus generalizing to this infinite-dimensional framework some classical results on real-valued stationary processes. …
The approximation of a stationary time-series by finite order autoregressive (AR) and moving averages (MA) is a problem that occurs in many applications. In this paper we study asymptotic behavior of the spectral density of finite order approximations of wide sense stationary time series. It is shown that when the on the spectral density is non-vanishing in $[-π,π]$ and the covariance is summable, the spectral density of the approximating autoregressive sequence converges at the origin. Under additional mild conditions on the coefficients of the Wold decomposition it is also shown that the spe…
This paper deals with the comparison of several stationary processes with unequal sample sizes. We provide a detailed theoretical framework on the testing problem for equality of spectral densities in the bivariate case, after which the generalization of our approach to the m dimensional case and to other statistical applications (like testing for zero correlation or clustering of time series data with different length) is straightforward. We prove asymptotic normality of an appropriately standardized version of the test statistic both under the null and the alternative and investigate the fin…
The aim of this paper is to give a simpler, more usable sufficient condition to the regularity of generic weakly stationary time series. Also, this condition is used to show how regular processes satisfying these sufficient conditions can be approximated by a lower rank \emph{regular} process. The relevance of these issues is shown by the ever increasing presence of high-dimensional data in many fields lately, and because of this, low rank processes and low rank approximations are becoming more important. Moreover, regular processes are the ones which are completely influenced by random innova…
Due to the surge of data storage techniques, the need for the development of appropriate techniques to identify patterns and to extract knowledge from the resulting enormous data sets, which can be viewed as collections of dependent functional data, is of increasing interest in many scientific areas. We develop a similarity measure for spectral density operators of a collection of functional time series, which is based on the aggregation of Hilbert-Schmidt differences of the individual time-varying spectral density operators. Under fairly general conditions, the asymptotic properties of the co…
We adapt the classical definition of locally stationary processes in discrete-time to the continuous-time setting and obtain equivalent representations in the time and frequency domain. From this, a unique time-varying spectral density is derived using the Wigner-Ville spectrum. As an example, we investigate time-varying Lévy-driven state space processes, including the class of time-varying Lévy-driven CARMA processes. First, the connection between these two classes of processes is examined. Considering a sequence of time-varying Lévy-driven state space processes, we then give sufficient condi…
We study efficiency of non-parametric estimation of diffusions (stochastic differential equations driven by Brownian motion) from long stationary trajectories. First, we introduce estimators based on conditional expectation which is motivated by the definition of drift and diffusion coefficients. These estimators involve time- and space-discretization parameters for computing expected values from discretely-sampled stationary data. Next, we analyze consistency and mean squared error of these estimators depending on computational parameters. We derive relationships between the number of observa…
Norwegian University of Science and Technology（挪威科技大学）；BI Norwegian Business School（BI挪威商学院）；Norwegian Seismic Array（挪威地震台阵）
One of the latest self-supervised learning (SSL) methods, VICReg, showed a great performance both in the linear evaluation and the fine-tuning evaluation. However, VICReg is proposed in computer vision and it learns by pulling representations of random crops of an image while maintaining the representation space by the variance and covariance loss. However, VICReg would be ineffective on non-stationary time series where different parts/crops of input should be differently encoded to consider the non-stationarity. Another recent SSL proposal, Temporal Neighborhood Coding (TNC) is effective for …
Accurate forecasting of the U.K. gross value added (GVA) is fundamental for measuring the growth of the U.K. economy. A common nonstationarity in GVA data, such as the ABML series, is its increase in variance over time due to inflation. Transformed or inflation-adjusted series can still be challenging for classical stationarity-assuming forecasters. We adopt a different approach that works directly with the GVA series by advancing recent forecasting methods for locally stationary time series. Our approach results in more accurate and reliable forecasts, and continues to work well even when the…
Methods of estimation and forecasting for stationary models are well known in classical time series analysis. However, stationarity is an idealization which, in practice, can at best hold as an approximation, but for many time series may be an unrealistic assumption. We define a class of locally stationary processes which can lead to more accurate uncertainty quantification over making an invalid assumption of stationarity. This class of processes assumes the model parameters to be time-varying and parameterizes them in terms of a transformation of basis functions that ensures that the process…
The Model-free Prediction Principle of Politis (2015) has been successfully applied to general regression problems, as well as problems involving stationary time series. However, with long time series, e.g. annual temperature measurements spanning over 100 years or daily financial returns spanning several years, it may be unrealistic to assume stationarity throughout the span of the dataset. In the paper at hand, we show how Model-free Prediction can be applied to handle time series that are only locally stationary, i.e., they can be assumed to be as stationary only over short time-windows. Su…
We develop an anomaly-detection method when systematic anomalies, possibly statistically very similar to genuine inputs, are affecting control systems at the input and/or output stages. The method allows anomaly-free inputs (i.e., those before contamination) to originate from a wide class of random sequences, thus opening up possibilities for diverse applications. To illustrate how the method works on data, and how to interpret its results and make decisions, we analyze several actual time series, which are originally non-stationary but in the process of analysis are converted into stationary.…
Bootstrap methods for estimating the long-run covariance of stationary functional time series are considered. We introduce a versatile bootstrap method that relies on functional principal component analysis, where principal component scores can be bootstrapped by maximum entropy. Two other bootstrap methods resample error functions, after the dependence structure being modeled linearly by a sieve method or nonlinearly by a functional kernel regression. Through a series of Monte-Carlo simulation, we evaluate and compare the finite-sample performances of these three bootstrap methods for estimat…
Testing for nonlinearity is one of the most important preprocessing steps in nonlinear time series analysis. Typically, this is done by means of the linear surrogate data methods. But it is a known fact that the validity of the results heavily depends on the stationarity of the time series. Since most physiological signals are non-stationary, it is easy to falsely detect nonlinearity using the linear surrogate data methods. In this document, we propose a methodology to extend the procedure for generating constrained surrogate time series in order to assess nonlinearity in non-stationary data. …
With developing of computation tools in the last years, data analysis methods to find insightful information are becoming more common among industries and researchers. This paper is the first part of the times series analysis of New England electricity price and demand to find anomaly in the data. In this paper time-series stationary criteria to prepare data for further times-series related analysis is investigated. Three main analysis are conducted in this paper, including moving average, moving standard deviation and augmented Dickey-Fuller test. The data used in this paper is New England bi…
Stochastic optimization naturally arises in machine learning. Efficient algorithms with provable guarantees, however, are still largely missing, when the objective function is nonconvex and the data points are dependent. This paper studies this fundamental challenge through a streaming PCA problem for stationary time series data. Specifically, our goal is to estimate the principle component of time series data with respect to the covariance matrix of the stationary distribution. Computationally, we propose a variant of Oja's algorithm combined with downsampling to control the bias of the stoch…
Monitoring economic conditions and financial stability with an early warning system serves as a prevention mechanism for unexpected economic events. In this paper, we investigate the statistical performance of sequential break-point detectors for stationary time series regression models with extensive simulation experiments. We employ an online sequential scheme for monitoring economic indicators from the European as well as the American financial markets that span the period during the 2008 financial crisis. Our results show that the performance of these tests applied to stationary time serie…
Time series with long-term structure arise in a variety of contexts and capturing this temporal structure is a critical challenge in time series analysis for both inference and forecasting settings. Traditionally, state space models have been successful in providing uncertainty estimates of trajectories in the latent space. More recently, deep learning, attention-based approaches have achieved state of the art performance for sequence modeling, though often require large amounts of data and parameters to do so. We propose Stanza, a nonlinear, non-stationary state space model as an intermediate…
In this note we show that the locally stationary wavelet process can be decomposed into a sum of signals, each of which following a moving average process with time-varying parameters. We then show that such moving average processes are equivalent to state space models with stochastic design components. Using a simple simulation step, we propose a heuristic method of estimating the above state space models and then we apply the methodology to foreign exchange rates data.
This paper proposes some novel one-sided omnibus tests for independence between two multivariate stationary time series. These new tests apply the Hilbert-Schmidt independence criterion (HSIC) to test the independence between the innovations of both time series. Under regular conditions, the limiting null distributions of our HSIC-based tests are established. Next, our HSIC-based tests are shown to be consistent. Moreover, a residual bootstrap method is used to obtain the critical values for our HSIC-based tests, and its validity is justified. Compared with the existing cross-correlation-based…
Entropy measures have become increasingly popular as an evaluation metric for complexity in the analysis of time series data, especially in physiology and medicine. Entropy measures the rate of information gain, or degree of regularity in a time series e.g. heartbeat. Ideally, entropy should be able to quantify the complexity of any underlying structure in the series, as well as determine if the variation arises from a random process. Unfortunately current entropy measures mostly are unable to perform the latter differentiation. Thus, a high entropy score indicates a random or chaotic series, …
We study causality between bivariate curve time series using the Granger causality generalized measures of correlation. With this measure, we can investigate which curve time series Granger-causes the other; in turn, it helps determine the predictability of any two curve time series. Illustrated by a climatology example, we find that the sea surface temperature Granger-causes the sea-level atmospheric pressure. Motivated by a portfolio management application in finance, we single out those stocks that lead or lag behind Dow-Jones industrial averages. Given a close relationship between S&P 500 …
To capture the slowly time-varying spectral content of real-world time-series, a common paradigm is to partition the data into approximately stationary intervals and perform inference in the time-frequency domain. However, this approach lacks a corresponding nonstationary time-domain generative model for the entire data and thus, time-domain inference occurs in each interval separately. This results in distortion/discontinuity around interval boundaries and can consequently lead to erroneous inferences based on any quantities derived from the posterior, such as the phase. To address these shor…
One of the most useful tools for distinguishing between chaotic and stochastic time series is the so-called complexity-entropy causality plane. This diagram involves two complexity measures: the Shannon entropy and the statistical complexity. Recently, this idea has been generalized by considering the Tsallis monoparametric generalization of the Shannon entropy, yielding complexity-entropy curves. These curves have proven to enhance the discrimination among different time series related to stochastic and chaotic processes of numerical and experimental nature. Here we further explore these comp…
In the traditional framework of spectral learning of stochastic time series models, model parameters are estimated based on trajectories of fully recorded observations. However, real-world time series data often contain missing values, and worse, the distributions of missingness events over time are often not independent of the visible process. Recently, a spectral OOM learning algorithm for time series with missing data was introduced and proved to be consistent, albeit under quite strong conditions. Here we refine the algorithm and prove that the original strong conditions can be very much r…
