Errors-in-variables regression for dependent data with estimated error covariance matrix: To prewhiten or not?
Guanghua School of Management, Peking University（北京大学光华管理学院）；Center for Statistical Science, Peking University（北京大学统计科学中心）；Department of Statistics and Data Science, Tsinghua University（清华大学统计学与数据科学系）
We consider statistical inference for errors-in-variables regression models with dependent observations under the high dimensionality of the error covariance matrix. It is tempting to prewhiten the model and data that had led to efficient weighted least squares estimation in the presence of the measurement errors, as being practised in the optimal fingerprinting approach in climate change studies. However, it is unclear to what extent the prewhitened estimator can improve the estimation efficiency of the unprewhitened estimator for errors-in-variables regression. We compare the prewhitening and unprewhitening estimators in terms of their estimation efficiency and computational cost. It shows that while the prewhitening operation does not necessarily improve the estimation efficiency of its unprewhitening counterpart, it demands more on the ensemble size needed in the error-covariance matrix estimation to ensure the asymptotic normality, and hence it would requires much more computationally resource.
In this paper we introduce a covariance framework for the analysis of EEG and MEG data that takes into account observed temporal stationarity on small time scales and trial-to-trial variations. We formulate a model for the covariance matrix, which is a Kronecker product of three components that correspond to space, time and epochs/trials, and consider maximum likelihood estimation of the unknown parameter values. An iterative algorithm that finds approximations of the maximum likelihood estimates is proposed. We perform a simulation study to assess the performance of the estimator and investig…
While most of the convergence results in the literature on high dimensional covariance matrix are concerned about the accuracy of estimating the covariance matrix (and precision matrix), relatively less is known about the effect of estimating large covariances on statistical inferences. We study two important models: factor analysis and panel data model with interactive effects, and focus on the statistical inference and estimation efficiency of structural parameters based on large covariance estimators. For efficient estimation, both models call for a weighted principle components (WPC), whic…
This article provides an introduction to the asymptotic analysis of covariance parameter estimation for Gaussian processes. Maximum likelihood estimation is considered. The aim of this introduction is to be accessible to a wide audience and to present some existing results and proof techniques from the literature. The increasing-domain and fixed-domain asymptotic settings are considered. Under increasing-domain asymptotics, it is shown that in general all the components of the covariance parameter can be estimated consistently by maximum likelihood and that asymptotic normality holds. In contr…
The assumption of separability is a simplifying and very popular assumption in the analysis of spatio-temporal or hypersurface data structures. It is often made in situations where the covariance structure cannot be easily estimated, for example because of a small sample size or because of computational storage problems. In this paper we propose a new and very simple test to validate this assumption. Our approach is based on a measure of separability which is zero in the case of separability and positive otherwise. The measure can be estimated without calculating the full non-separable covaria…
Modeling and forecasting covariance matrices of asset returns play a crucial role in finance. The availability of high frequency intraday data enables the modeling of the realized covariance matrix directly. However, most models in the literature suffer from the curse of dimensionality. To solve the problem, we propose a factor model with a diagonal CAW model for the factor realized covariance matrices. Asymptotic theory is derived for the estimated parameters. In an extensive empirical analysis, we find that the number of parameters can be reduced significantly. Furthermore, the proposed mode…
The estimation of covariance matrices of gene expressions has many applications in cancer systems biology. Many gene expression studies, however, are hampered by low sample size and it has therefore become popular to increase sample size by collecting gene expression data across studies. Motivated by the traditional meta-analysis using random effects models, we present a hierarchical random covariance model and use it for the meta-analysis of gene correlation networks across 11 large-scale gene expression studies of diffuse large B-cell lymphoma (DLBCL). We suggest to use a maximum likelihood …
The association between two random variables is often of primary interest in statistical research. In this paper semiparametric models for the association between random vectors X and Y are considered which leave the marginal distributions arbitrary. Given that the odds ratio function comprises the whole information about the association the focus is on bilinear log-odds ratio models and in particular on the odds ratio parameter vector θ. The covariance structure of the maximum likelihood estimator θ^ of θ is of major importance for asymptotic inference. To this end different representations o…
In this paper, we propose a new test for the equality of several covariance functions for functional data. Its test statistic is taken as the supremum value of the sum of the squared differences between the estimated individual covariance functions and the pooled sample covariance function, hoping to obtain a more powerful test than some existing tests for the same testing problem. The asymptotic random expression of this test statistic under the null hypothesis is obtained. To approximate the null distribution of the proposed test statistic, we describe a parametric bootstrap method and a non…
We consider goodness-of-fit tests of symmetric stable distributions based on weighted integrals of the squared distance between the empirical characteristic function of the standardized data and the characteristic function of the standard symmetric stable distribution with the characteristic exponent $α$ estimated from the data. We treat $α$ as an unknown parameter, but for theoretical simplicity we also consider the case that $α$ is fixed. For estimation of parameters and the standardization of data we use maximum likelihood estimator (MLE) and an equivariant integrated squared error estimato…
In many situations, when dealing with several populations, equality of the covariance operators is assumed. An important issue is to study if this assumption holds before making other inferences. In this paper, we develop a test for comparing covariance operators of several functional data samples. The proposed test is based on the Hilbert--Schmidt norm of the difference between estimated covariance operators. In particular, when dealing with two populations, the tests statistic is just the squared norm of the difference between the two covariance operators estimators. The asymptotic behaviour…
There has been a growing interest in providing models for multivariate spatial processes. A majority of these models specify a parametric matrix covariance function. Based on observations, the parameters are estimated by maximum likelihood or variants thereof. While the asymptotic properties of maximum likelihood estimators for univariate spatial processes have been analyzed in detail, maximum likelihood estimators for multivariate spatial processes have not received their deserved attention yet. In this article we consider the classical increasing-domain asymptotic setting restricting the min…
Estimation of the covariance structure of spatial processes is of fundamental importance in spatial statistics. In the literature, several non-parametric and semi-parametric methods have been developed to estimate the covariance structure based on the spectral representation of covariance functions. However,they either ignore the high frequency properties of the spectral density, which are essential to determine the performance of interpolation procedures such as Kriging, or lack of theoretical justification. We propose a new semi-parametric method to estimate spectral densities of isotropic s…
The assumption of separability of the covariance operator for a random image or hypersurface can be of substantial use in applications, especially in situations where the accurate estimation of the full covariance structure is unfeasible, either for computational reasons, or due to a small sample size. However, inferential tools to verify this assumption are somewhat lacking in high-dimensional or functional {data analysis} settings, where this assumption is most relevant. We propose here to test separability by focusing on $K$-dimensional projections of the difference between the covariance o…
When considering two or more time series of functions or curves, for instance those derived from densely observed intraday stock price data of several companies, the empirical cross-covariance operator is of fundamental importance due to its role in functional lagged regression and exploratory data analysis. Despite its relevance, statistical procedures for measuring the significance of such estimators are undeveloped. We present methodology based on a functional central limit theorem for conducting statistical inference for the cross-covariance operator estimated between two stationary, weakl…
Physics-based covariance models provide a systematic way to construct covariance models that are consistent with the underlying physical laws in Gaussian process analysis. The unknown parameters in the covariance models can be estimated using maximum likelihood estimation, but direct construction of the covariance matrix and classical strategies of computing with it requires $n$ physical model runs, $n^2$ storage complexity, and $n^3$ computational complexity. To address such challenges, we propose to approximate the discretized covariance function using hierarchical matrices. By utilizing ran…
Channel covariance is emerging as a critical ingredient of the acquisition of instantaneous channel state information (CSI) in multi-user Massive MIMO systems operating in frequency division duplex (FDD) mode. In this context, channel reciprocity does not hold, and it is generally expected that covariance information about the downlink channel must be estimated and fed back by the user equipment (UE). As an alternative CSI acquisition technique, we propose to infer the downlink covariance based on the observed uplink covariance. This inference process relies on a dictionary of uplink/downlink …
The radio astronomy community is currently building a number of phased array telescopes. The calibration of these telescopes is hampered by the fact that covariances of signals from closely spaced antennas are sensitive to noise coupling and to variations in sky brightness on large spatial scales. These effects are difficult and computationally expensive to model. We propose to model them phenomenologically using a non-diagonal noise covariance matrix. The parameters can be estimated using a weighted alternating least squares (WALS) algorithm iterating between the calibration parameters and th…
This paper studies methods for testing and estimating change-points in the covariance structure of a high-dimensional linear time series. The assumed framework allows for a large class of multivariate linear processes (including vector autoregressive moving average (VARMA) models) of growing dimension and spiked covariance models. The approach uses bilinear forms of the centered or non-centered sample variance-covariance matrix. Change-point testing and estimation are based on maximally selected weighted cumulated sum (CUSUM) statistics. Large sample approximations under a change-point regime …
This paper examines the usefulness of high frequency data in estimating the covariance matrix for portfolio choice when the portfolio size is large. A computationally convenient nonlinear shrinkage estimator for the integrated covariance (ICV) matrix of financial assets is developed in two steps. The eigenvectors of the ICV are first constructed from a designed time variation adjusted realized covariance matrix of noise-free log-returns of relatively low frequency data. Then the regularized eigenvalues of the ICV are estimated by quasi-maximum likelihood based on high frequency data. The estim…
Omics technologies are powerful tools for analyzing patterns in gene expression data for thousands of genes. Due to a number of systematic variations in experiments, the raw gene expression data is often obfuscated by undesirable technical noises. Various normalization techniques were designed in an attempt to remove these non-biological errors prior to any statistical analysis. One of the reasons for normalizing data is the need for recovering the covariance matrix used in gene network analysis. In this paper, we introduce a novel normalization technique, called the covariance shift (C-SHIFT)…
