We propose a difference-based nonparametric methodology for the estimation and inference of the time-varying auto-covariance functions of a locally stationary time series when it is contaminated by a complex trend with both abrupt and smooth changes. Simultaneous confidence bands (SCB) with asymptotically correct coverage probabilities are constructed for the auto-covariance functions under complex trend. A simulation-assisted bootstrapping method is proposed for the practical construction of the SCB. Detailed simulation and a real data example round out our presentation.
For multivariate stationary time series many important properties, such as partial correlation, graphical models and autoregressive representations are encoded in the inverse of its spectral density matrix. This is not true for nonstationary time series, where the pertinent information lies in the inverse infinite dimensional covariance matrix operator associated with the multivariate time series. This necessitates the study of the covariance of a multivariate nonstationary time series and its relationship to its inverse. We show that if the rows/columns of the infinite dimensional covariance …
For testing hypothesis on the covariance operator of functional time series, we suggest to use the full functional information and to avoid dimension reduction techniques. The limit distribution follows from the central limit theorem of the weak convergence of the partial sum process in general Hilbert space applied to the product space. In order to obtain critical values for tests, we generalize bootstrap results from the independent to the dependent case. This results can be applied to covariance operators, autocovariance operators and cross covariance operators. We discuss one sample and ch…
To derive the auto-covariance function from a sampled and time-limited signal or the cross-covariance function from two such signals, the mean values must be estimated and removed from the signals. If no a priori information about the correct mean values is available and the mean values must be derived from the time series themselves, the estimates will be biased. For the estimation of the variance from independent data the appropriate correction is widely known as Bessel's correction. Similar corrections for the auto-covariance and for the cross-covariance functions are shown here, including …
We develop an estimator for the high-dimensional covariance matrix of a locally stationary process with a smoothly varying trend and use this statistic to derive consistent predictors in non-stationary time series. In contrast to the currently available methods for this problem the predictor developed here does not rely on fitting an autoregressive model and does not require a vanishing trend. The finite sample properties of the new methodology are illustrated by means of a simulation study and a financial indices study.
We complement the theory developed in Preinerstorfer and Pötscher (2016) with further finite sample results on size and power of heteroskedasticity and autocorrelation robust tests. These allows us, in particular, to show that the sufficient conditions for the existence of size-controlling critical values recently obtained in Pötscher and Preinerstorfer (2018) are often also necessary. We furthermore apply the results obtained to tests for hypotheses on deterministic trends in stationary time series regressions, and find that many tests currently used are strongly size-distorted.
In many statistical modeling frameworks, goodness-of-fit tests are typically administered to the estimated residuals. In the time series setting, whiteness of the residuals is assessed using the sample autocorrelation function. For many time series models, especially those used for financial time series, the key assumption on the residuals is that they are in fact independent and not just uncorrelated. In this paper, we apply the auto-distance covariance function (ADCV) to evaluate the serial dependence of the estimated residuals. Distance covariance can discriminate between dependence and ind…
We clarify relationships between conditional (CAR) and simultaneous (SAR) autoregressive models. We review the literature on this topic and find that it is mostly incomplete. Our main result is that a SAR model can be written as a unique CAR model, and while a CAR model can be written as a SAR model, it is not unique. In fact, we show how any multivariate Gaussian distribution on a finite set of points with a positive-definite covariance matrix can be written as either a CAR or a SAR model. We illustrate how to obtain any number of SAR covariance matrices from a single CAR covariance matrix by…
A causal vector autoregressive (CVAR) model is introduced for weakly stationary multivariate processes, combining a recursive directed graphical model for the contemporaneous components and a vector autoregressive model longitudinally. Block Cholesky decomposition with varying block sizes is used to solve the model equations and estimate the path coefficients along a directed acyclic graph (DAG). If the DAG is decomposable, i.e. the zeros form a reducible zero pattern (RZP) in its adjacency matrix, then covariance selection is applied that assigns zeros to the corresponding path coefficients. …
The link between Gaussian random fields and Markov random fields is well established based on a stochastic partial differential equation in Euclidean spaces, where the Matérn covariance functions are essential. However, the Matérn covariance functions are not always positive definite on circles and spheres. In this manuscript, we focus on the extension of this link to circles, and show that the link between Gaussian random fields and Markov random fields on circles is valid based on the circular Matérn covariance function instead. First, we show that this circular Matérn function is the covari…
The behavior of the power function of autocorrelation tests such as the Durbin-Watson test in time series regressions or the Cliff-Ord test in spatial regression models has been intensively studied in the literature. When the correlation becomes strong, Krämer (1985) (for the Durbin-Watson test) and Krämer (2005) (for the Cliff-Ord test) have shown that the power can be very low, in fact can converge to zero, under certain circumstances. Motivated by these results, Martellosio (2010) set out to build a general theory that would explain these findings. Unfortunately, Martellosio (2010) does not…
We begin with an interpretation of the L1-distance between two power spectral densities and then, following an analogous rationale, we develop a natural metric for quantifying distance between respective covariance matrices.
This paper studies methods for testing and estimating change-points in the covariance structure of a high-dimensional linear time series. The assumed framework allows for a large class of multivariate linear processes (including vector autoregressive moving average (VARMA) models) of growing dimension and spiked covariance models. The approach uses bilinear forms of the centered or non-centered sample variance-covariance matrix. Change-point testing and estimation are based on maximally selected weighted cumulated sum (CUSUM) statistics. Large sample approximations under a change-point regime …
Objective: Electroencephalography signals are recorded as a multidimensional dataset. We propose a new framework based on the augmented covariance extracted from an autoregressive model to improve motor imagery classification. Methods: From the autoregressive model can be derived the Yule-Walker equations, which show the emergence of a symmetric positive definite matrix: the augmented covariance matrix. The state-of the art for classifying covariance matrices is based on Riemannian Geometry. A fairly natural idea is therefore to extend the standard approach using these augmented covariance mat…
It is well recognised that animal and plant pathogens form complex ecological communities of interacting organisms within their hosts. Although community ecology approaches have been applied to determine pathogen interactions at the within-host scale, methodologies enabling robust inference of the epidemiological impact of pathogen interactions are lacking. Here we developed a novel statistical framework to identify statistical covariances from the infection time-series of multiple pathogens simultaneously. Our framework extends Bayesian multivariate disease mapping models to analyse multivari…
Fractional lower-order covariance-based measures for cyclostationary time series with heavy-tailed distributions: application to dependence testing and model order identification
Faculty of Pure and Applied Mathematics, Hugo Steinhaus Center, Wroclaw University of Science and Technology（弗罗茨瓦夫科技大学纯科学与应用数学学院，雨果·斯坦豪斯中心）
We consider a class of vector autoregressive models with banded coefficient matrices. The setting represents a type of sparse structure for high-dimensional time series, though the implied autocovariance matrices are not banded. The structure is also practically meaningful when the order of component time series is arranged appropriately. The convergence rates for the estimated banded autoregressive coefficient matrices are established. We also propose a Bayesian information criterion for determining the width of the bands in the coefficient matrices, which is proved to be consistent. By explo…
I present an approach for modeling areal spatial covariance by considering the stationary distribution of a spatio-temporal Markov random walk. In the areal data case, this stationary distribution corresponds to an intrinsic simultaneous autoregressive (SAR) model for spatial correlation, and provides a principled approach to specifying areal spatial models when a spatio-temporal generating process can be assumed. I apply the approach to a study of spatial genetic variation of trout in a stream network in Connecticut, USA, and a study of crime rates in neighborhoods of Columbus, OH, USA.
In Jin et al. (2014), the limiting spectral distribution (LSD) of a symmetrized auto-cross covariance matrix is derived using matrix manipulation, with finite $(2+δ)$-th moment assumption. Here we give an alternative method using a result in Bai and Silverstein (2010), in which a weaker condition of finite 2nd moment is assumed.
We present a novel quasi-Bayesian method to weight multiple dynamical models by their skill at capturing both potentially non-linear trends and first-order autocorrelated variability of the underlying process, and to make weighted probabilistic projections. We validate the method using a suite of one-at-a-time cross-validation experiments involving Atlantic meridional overturning circulation (AMOC), its temperature-based index, as well as Korean summer mean maximum temperature. In these experiments the method tends to exhibit superior skill over a trend-only Bayesian model averaging weighting …
We present a Bayesian model for area-level count data that uses Gaussian random effects with a novel type of G-Wishart prior on the inverse variance--covariance matrix. Specifically, we introduce a new distribution called the truncated G-Wishart distribution that has support over precision matrices that lead to positive associations between the random effects of neighboring regions while preserving conditional independence of non-neighboring regions. We describe Markov chain Monte Carlo sampling algorithms for the truncated G-Wishart prior in a disease mapping context and compare our results t…
Covariance matrix tapers have a long history in signal processing and related fields. Examples of applications include autoregressive models (promoting a banded structure) or beamforming (widening the spectral null width associated with an interferer). In this paper, the focus is on high-dimensional setting where the dimension $p$ is high, while the data aspect ratio $n/p$ is low. We propose an estimator called Tabasco (TApered or BAnded Shrinkage COvariance matrix) that shrinks the tapered sample covariance matrix towards a scaled identity matrix. We derive optimal and estimated (data adaptiv…
Concurrent time series commonly arise in various applications, including when monitoring the environment such as in air quality measurement networks, weather stations, oceanographic buoys, or in paleo form such as lake sediments, tree rings, ice cores, or coral isotopes, with each monitoring or sampling site providing one of the time series. The goal in such applications is to extract a common time trend or signal in the observed data. Other examples where the goal is to extract a common time trend for multiple time series are in stock price time series, neurological time series, and quality c…
The concept of distance covariance/correlation was introduced recently to characterize dependence among vectors of random variables. We review some statistical aspects of distance covariance/correlation function and we demonstrate its applicability to time series analysis. We will see that the auto-distance covariance/correlation function is able to identify nonlinear relationships and can be employed for testing the i.i.d.\ hypothesis. Comparisons with other measures of dependence are included.
Probabilistic programming languages represent complex data with intermingled models in a few lines of code. Efficient inference algorithms in probabilistic programming languages make possible to build unified frameworks to compute interesting probabilities of various large, real-world problems. When the structure of model is given, constructing a probabilistic program is rather straightforward. Thus, main focus have been to learn the best model parameters and compute marginal probabilities. In this paper, we provide a new perspective to build expressive probabilistic program from continue time…
We consider a stochastic process model with time trend and measurement error. We establish consistency and derive the limiting distributions of the maximum likelihood (ML) estimators of the covariance function parameters under a general asymptotic framework, including both the fixed domain and the increasing domain frameworks, even when the time trend model is misspecified or its complexity increases with the sample size. In particular, the convergence rates of the ML estimators are thoroughly characterized in terms of the growing rate of the domain and the degree of model misspecification/com…
A four-parameter family of covariance functions for stationary Gaussian processes is presented. We call it 2Dsys. It corresponds to the general solution of an autonomous second-order linear stochastic differential equation, thus arises naturally from modelling. It covers underdamped and overdamped systems, so it is proposed to use this family when one wishes to decide if a time-series corresponds to stochastically forced damped oscillations or a stochastically forced overdamped system.
The auto-cross covariance matrix is defined as \[\mathbf{M}_n=\frac{1} {2T}\sum_{j=1}^T\bigl(\mathbf{e}_j\mathbf{e}_{j+τ}^*+\mathbf{e}_{j+ τ}\mathbf{e}_j^*\bigr),\] where $\mathbf{e}_j$'s are $n$-dimensional vectors of independent standard complex components with a common mean 0, variance $σ^2$, and uniformly bounded $2+η$th moments and $τ$ is the lag. Jin et al. [Ann. Appl. Probab. 24 (2014) 1199-1225] has proved that the LSD of $\mathbf{M}_n$ exists uniquely and nonrandomly, and independent of $τ$ for all $τ\ge 1$. And in addition they gave an analytic expression of the LSD. As a continuatio…
Classical spectral analysis is based on the discrete Fourier transform of the auto-covariances. In this paper we investigate the asymptotic properties of new frequency domain methods where the auto-covariances in the spectral density are replaced by alternative dependence measures which can be estimated by U-statistics. An interesting example is given by Kendall{'}s $τ$ , for which the limiting variance exhibits a surprising behavior.
We propose nonparametric estimators for the second-order central moments of possibly anisotropic spherical random fields, within a functional data analysis context. We consider a measurement framework where each random field among an identically distributed collection of spherical random fields is sampled at a few random directions, possibly subject to measurement error. The collection of random fields could be i.i.d. or serially dependent. Though similar setups have already been explored for random functions defined on the unit interval, the nonparametric estimators proposed in the literature…
This article is devoted to study the effects of the S-periodical fractional differencing filter $(1-L^S)^{D_t}$. To put this effect in evidence, we have derived the periodic auto-covariance functions of two distinct univariate seasonally fractionally differenced periodic models. A multivariate representation of periodically correlated process is exploited to provide the exact and approximated expression auto-covariance of each models. The distinction between the models is clearly obvious through the expression of periodic autocovariance function. Besides producing different autocovariance func…
Lag windows are commonly used in time series, econometrics, steady-state simulation, and Markov chain Monte Carlo to estimate time-average covariance matrices. In the presence of positive correlation of the underlying process, estimators of this matrix almost always exhibit significant negative bias, leading to undesirable finite-sample properties. We propose a new family of lag windows specifically designed to improve finite-sample performance by offsetting this negative bias. Any existing lag window can be adapted into a lugsail equivalent with no additional assumptions. We use these lag win…
We elucidate the problem of estimating large-dimensional covariance matrices in the presence of correlations between samples. To this end, we generalize the Marcenko-Pastur equation and the Ledoit-Peche shrinkage estimator using methods of random matrix theory and free probability. We develop an efficient algorithm that implements the corresponding analytic formulas, based on the Ledoit-Wolf kernel estimation technique. We also provide an associated open-source Python library, called "shrinkage", with a user-friendly API to assist in practical tasks of estimation of large covariance matrices. …
We offer a survey of recent results on covariance estimation for heavy-tailed distributions. By unifying ideas scattered in the literature, we propose user-friendly methods that facilitate practical implementation. Specifically, we introduce element-wise and spectrum-wise truncation operators, as well as their $M$-estimator counterparts, to robustify the sample covariance matrix. Different from the classical notion of robustness that is characterized by the breakdown property, we focus on the tail robustness which is evidenced by the connection between nonasymptotic deviation and confidence le…
The non-parametric estimation of covariance lies at the heart of functional data analysis, whether for curve or surface-valued data. The case of a two-dimensional domain poses both statistical and computational challenges, which are typically alleviated by assuming separability. However, separability is often questionable, sometimes even demonstrably inadequate. We propose a framework for the analysis of covariance operators of random surfaces that generalises separability, while retaining its major advantages. Our approach is based on the expansion of the covariance into a series of separable…
