Limit theorems are proved for quadratic forms of Gaussian random fields in presence of long memory. We obtain a non central limit theorem under a minimal integrability condition, which allows isotropic and anisotropic models. We apply our limit theorems and those of Ginovian (99) to obtain the asymptotic behavior of the empirical covariances of Gaussian fields, which is a particular example of quadratic forms. We show that it is possible to obtain a Gaussian limit when the spectral density is not in $L^2$. Therefore the dichotomy observed in dimension $d=1$ between central and non central limit theorems cannot be stated so easily due to possible anisotropic strong dependence in $d>1$.
This paper presents a central limit theorem for a pre-averaged version of the realized covariance estimator for the quadratic covariation of a discretely observed semimartingale with noise. The semimartingale possibly has jumps, while the observation times show irregularity, non-synchronicity, and some dependence on the observed process. It is shown that the observation times' effect on the asymptotic distribution of the estimator is only through two characteristics: the observation frequency and the covariance structure of the noise. This is completely different from the case of the realized …
A central limit theorem (CLT) for the smoothed empirical spectral distribution of sample covariance matrices is established. Moreover, the CLTs for the smoothed quantiles of Marcenko and Pastur's law have been also developed.
We give two asymptotic results for the empirical distance covariance on separable metric spaces without any iid assumption on the samples. In particular, we show the almost sure convergence of the empirical distance covariance for any measure with finite first moments, provided that the samples form a strictly stationary and ergodic process. We further give a result concerning the asymptotic distribution of the empirical distance covariance under the assumption of absolute regularity of the samples and extend these results to certain types of pseudometric spaces. In the process, we derive a ge…
Let $X,X_1,\dots, X_n$ be i.i.d. Gaussian random variables with zero mean and covariance operator $Σ={\mathbb E}(X\otimes X)$ taking values in a separable Hilbert space ${\mathbb H}.$ Let $$ {\bf r}(Σ):=\frac{{\rm tr}(Σ)}{\|Σ\|_{\infty}} $$ be the effective rank of $Σ,$ ${\rm tr}(Σ)$ being the trace of $Σ$ and $\|Σ\|_{\infty}$ being its operator norm. Let $$\hat Σ_n:=n^{-1}\sum_{j=1}^n (X_j\otimes X_j)$$ be the sample (empirical) covariance operator based on $(X_1,\dots, X_n).$ The paper deals with a problem of estimation of spectral projectors of the covariance operator $Σ$ by their empirical…
We propose a new estimator for the spot covariance matrix of a multi-dimensional continuous semi-martingale log asset price process which is subject to noise and non-synchronous observations. The estimator is constructed based on a local average of block-wise parametric spectral covariance estimates. The latter originate from a local method of moments (LMM) which recently has been introduced. We prove consistency and a point-wise stable central limit theorem for the proposed spot covariance estimator in a very general setup with stochastic volatility, leverage effects and general noise distrib…
The author uses a Stein-type covariance identity to obtain moment estimators for the parameters of the quadratic polynomial subfamily of Pearson distributions. The asymptotic distribution of the estimators is obtained, and normality and symmetry tests based on it are provided. Simulation is used to compare the performance of the proposed tests with that of other existing tests for symmetry and normality.
The problem of estimating the spectral density matrix $f(w)$ of a multivariate time series is revisited with special focus on the frequencies $w=0$ and $w=π$. Recognizing that the entries of the spectral density matrix at these two boundary points are real-valued, we propose a new estimator constructed from a local polynomial regression of the real portion of the multivariate periodogram. The case $w=0$ is of particular importance, since $f(0)$ is associated with the large-sample covariance matrix of the sample mean; hence, estimating $f(0)$ is crucial in order to conduct any sort of statistic…
The L1-regularized Gaussian maximum likelihood estimator (MLE) has been shown to have strong statistical guarantees in recovering a sparse inverse covariance matrix, or alternatively the underlying graph structure of a Gaussian Markov Random Field, from very limited samples. We propose a novel algorithm for solving the resulting optimization problem which is a regularized log-determinant program. In contrast to recent state-of-the-art methods that largely use first order gradient information, our algorithm is based on Newton's method and employs a quadratic approximation, but with some modific…
We consider the asymptotic normality in $L^2$ of kernel estimators of the long run covariance kernel of stationary functional time series. Our results are established assuming a weakly dependent Bernoulli shift structure for the underlying observations, which contains most stationary functional time series models, under mild conditions. As a corollary, we obtain joint asymptotics for functional principal components computed from empirical long run covariance operators, showing that they have the favorable property of being asymptotically independent.
We develop tests for high-dimensional covariance matrices under a generalized elliptical model. Our tests are based on a central limit theorem (CLT) for linear spectral statistics of the sample covariance matrix based on self-normalized observations. For testing sphericity, our tests neither assume specific parametric distributions nor involve the kurtosis of data. More generally, we can test against any non-negative definite matrix that can even be not invertible. As an interesting application, we illustrate in empirical studies that our tests can be used to test uncorrelatedness among idiosy…
We propose an independence test for random variables valued into metric spaces by using a test statistic obtained from appropriately centering and rescaling the squared Hilbert-Schmidt norm of the usual empirical estimator of normalized cross-covariance operator. We then get asymptotic normality of this statistic under independence hypothesis, so leading to a new test for independence of functional random variables. A simulation study that allows to compare the proposed test to existing ones is provided.
We propose three measures of mutual dependence between multiple random vectors. All the measures are zero if and only if the random vectors are mutually independent. The first measure generalizes distance covariance from pairwise dependence to mutual dependence, while the other two measures are sums of squared distance covariance. All the measures share similar properties and asymptotic distributions to distance covariance, and capture non-linear and non-monotone mutual dependence between the random vectors. Inspired by complete and incomplete V-statistics, we define the empirical measures and…
This article provides a central limit theorem for a consistent estimator of population eigenvalues with large multiplicities based on sample covariance matrices. The focus is on limited sample size situations, whereby the number of available observations is known and comparable in magnitude to the observation dimension. An exact expression as well as an empirical, asymptotically accurate, approximation of the limiting variance is derived. Simulations are performed that corroborate the theoretical claims. A specific application to wireless sensor networks is developed.
In this paper, we use the dimensional reduction technique to study the central limit theory (CLT) random quadratic forms based on sample means and sample covariance matrices. Specifically, we use a matrix denoted by $U_{p\times q}$, to map $q$-dimensional sample vectors to a $p$ dimensional subspace, where $q\geq p$ or $q\gg p$. Under the condition of $p/n\rightarrow 0$ as $(p,n)\rightarrow \infty$, we obtain the CLT of random quadratic forms for the sample means and sample covariance matrices.
We propose a new estimator, the quadratic form estimator, of the Kronecker product model for covariance matrices. We show that this estimator has good properties in the large dimensional case (i.e., the cross-sectional dimension $n$ is large relative to the sample size $T$). In particular, the quadratic form estimator is consistent in a relative Frobenius norm sense provided $\log^3n/T\to 0$. We obtain the limiting distributions of Lagrange multiplier (LM) and Wald tests under both the null and local alternatives concerning the mean vector $μ$. Testing linear restrictions of $μ$ is also invest…
We provide a general theoretical framework to derive Bernstein-von Mises theorems for matrix functionals. The conditions on functionals and priors are explicit and easy to check. Results are obtained for various functionals including entries of covariance matrix, entries of precision matrix, quadratic forms, log-determinant, eigenvalues in the Bayesian Gaussian covariance/precision matrix estimation setting, as well as for Bayesian linear and quadratic discriminant analysis.
Conditions are given under which the empirical copula process associated with a random sample from a bivariate continuous distribution has a smaller asymptotic covariance function than the standard empirical process based on observations from the copula. Illustrations are provided and consequences for inference are outlined.
Estimation of the mean vector and covariance matrix is of central importance in the analysis of multivariate data. In the framework of generalized linear models, usually the variances are certain functions of the means with the normal distribution being an exception. We study some implications of functional relationships between covariance and the mean by focusing on the maximum likelihood and Bayesian estimation of the mean-covariance under the joint constraint $\bmΣ\bmμ = \bmμ$ for a multivariate normal distribution. A novel structured covariance is proposed through reparameterization of the…
Moment inequality for quadratic forms of random vectors is of particular interest in covariance matrix testing and estimation problems. In this paper, we prove a Rosenthal-type inequality, which exhibits new features and certain improvement beyond the unstructured Rosenthal inequality of quadratic forms when dimension of the vectors increases without bound. Applications to test the block diagonal structures and detect the sparsity in the high-dimensional covariance matrix are presented.
Local covariance structure under the manifold setup has been widely applied in the machine learning society. Based on the established theoretical results, we provide an extensive study of two relevant manifold learning algorithms, empirical intrinsic geometry (EIG) and the locally linear embedding (LLE) under the manifold setup. Particularly, we show that without an accurate dimension estimation, the geodesic distance estimation by EIG might be corrupted. Furthermore, we show that by taking the local covariance matrix into account, we can more accurately estimate the local geodesic distance. W…
We establish large sample approximations for an arbitray number of bilinear forms of the sample variance-covariance matrix of a high-dimensional vector time series using $ \ell_1$-bounded and small $\ell_2$-bounded weighting vectors. Estimation of the asymptotic covariance structure is also discussed. The results hold true without any constraint on the dimension, the number of forms and the sample size or their ratios. Concrete and potential applications are widespread and cover high-dimensional data science problems such as tests for large numbers of covariances, sparse portfolio optimization…
The distance covariance of two random vectors is a measure of their dependence. The empirical distance covariance and correlation can be used as statistical tools for testing whether two random vectors are independent. We propose an analogs of the distance covariance for two stochastic processes defined on some interval. Their empirical analogs can be used to test the independence of two processes.
We establish that the limiting spectral distribution of a block-rescaled empirical covariance matrix is an arcsine law when the ratio between the dimension and the underlying sample size converges to 1 and when the samples corresponding to each block are independent. We further propose a conjecture for the cases where the latter ratio converges to a constant in the unit interval.
The eigenvector Empirical Spectral Distribution (VESD) is adopted to investigate the limiting behavior of eigenvectors and eigenvalues of covariance matrices. In this paper, we shall show that the Kolmogorov distance between the expected VESD of sample covariance matrix and the Marčenko-Pastur distribution function is of order $O(N^{-1/2})$. Given that data dimension $n$ to sample size $N$ ratio is bounded between 0 and 1, this convergence rate is established under finite 10th moment condition of the underlying distribution. It is also shown that, for any fixed $η>0$, the convergence rates of …
In this article we investigate high-dimensional banded sample covariance matrices under the regime that the sample size $n$, the dimension $p$ and the bandwidth $d$ tend simultaneously to infinity such that $$n/p\to 0 \ \ \text{and} \ \ 2d/n\to y>0.$$ It is shown that the empirical spectral distribution of those matrices almost surely converges weakly to some deterministic probability measure which is characterized by its moments. Certain restricted compositions of natural numbers play a crucial role in the evaluation of the expected moments of the empirical spectral distribution.
In the estimation of the mean matrix in a multivariate normal distribution, the generalized Bayes estimators with closed forms are provided, and the sufficient conditions for their minimaxity are derived relative to both matrix and scalar quadratic loss functions. The generalized Bayes estimators of the covariance matrix are also given with closed forms, and the dominance properties are discussed for the Stein loss function.
We consider high-dimensional quadratic classifiers in non-sparse settings. The target of classification rules is not Bayes error rates in the context. The classifier based on the Mahalanobis distance does not always give a preferable performance even if the populations are normal distributions having known covariance matrices. The quadratic classifiers proposed in this paper draw information about heterogeneity effectively through both the differences of expanding mean vectors and covariance matrices. We show that they hold a consistency property in which misclassification rates tend to zero a…
Testing large covariance matrices is of fundamental importance in statistical analysis with high-dimensional data. In the past decade, three types of test statistics have been studied in the literature: quadratic form statistics, maximum form statistics, and their weighted combination. It is known that quadratic form statistics would suffer from low power against sparse alternatives and maximum form statistics would suffer from low power against dense alternatives. The weighted combination methods were introduced to enhance the power of quadratic form statistics or maximum form statistics when…
Shrinkage estimators of covariance are an important tool in modern applied and theoretical statistics. They play a key role in regularized estimation problems, such as ridge regression (aka Tykhonov regularization), regularized discriminant analysis and a variety of optimization problems. In this paper, we bring to bear the tools of random matrix theory to understand their behavior, and in particular, that of quadratic forms involving inverses of those estimators, which are important in practice. We use very mild assumptions compared to the usual assumptions made in random matrix theory, requi…
We investigate an empirical Bayesian nonparametric approach to a family of linear inverse problems with Gaussian prior and Gaussian noise. We consider a class of Gaussian prior probability measures with covariance operator indexed by a hyperparameter that quantifies regularity. By introducing two auxiliary problems, we construct an empirical Bayes method and prove that this method can automatically select the hyperparameter. In addition, we show that this adaptive Bayes procedure provides optimal contraction rates up to a slowly varying term and an arbitrarily small constant, without knowledge…
We obtain a sharp convergence rate for banded covariance matrix estimates of stationary processes. A precise order of magnitude is derived for spectral radius of sample covariance matrices. We also consider a thresholded covariance matrix estimator that can better characterize sparsity if the true covariance matrix is sparse. As our main tool, we implement Toeplitz [Math. Ann. 70 (1911) 351-376] idea and relate eigenvalues of covariance matrices to the spectral densities or Fourier transforms of the covariances. We develop a large deviation result for quadratic forms of stationary processes us…
To explore the limits of a stochastic gradient method, it may be useful to consider an example consisting of an infinite number of quadratic functions. In this context, it is appropriate to determine the expected value and the covariance matrix of the stochastic noise, i.e. the difference of the true gradient and the approximated gradient generated from a finite sample. When specifying the covariance matrix, the expected value of a quadratic form QBQ is needed, where Q is a Wishart distributed random matrix and B is an arbitrary fixed symmetric matrix. After deriving an expression for E(QBQ) a…
We study distributional properties of a quadratic form of a stationary functional time series under mild moment conditions. As an important application, we obtain consistency rates of estimators of spectral density operators and prove joint weak convergence to a vector of complex Gaussian random operators. Weak convergence is established based on an approximation of the form via transforms of Hilbert-valued martingale difference sequences. As a side-result, the distributional properties of the long-run covariance operator are established.
We investigate the asymptotics of eigenvalues of sample covariance matrices associated with a class of non-independent Gaussian processes (separable and temporally stationary) under the Kolmogorov asymptotic regime. The limiting spectral distribution (LSD) is shown to depend explicitly on the Kolmogorov constant (a fixed limiting ratio of the sample size to the dimensionality) and parameters representing the spatio- and temporal- correlations. The Cauchy, M- and N-transforms from free harmonic analysis play key roles to this LSD calculation problem. The free multiplication law of free random v…
We derive the asymptotic distributions of the spiked eigenvalues and eigenvectors under a generalized and unified asymptotic regime, which takes into account the spike magnitude of leading eigenvalues, sample size, and dimensionality. This new regime allows high dimensionality and diverging eigenvalue spikes and provides new insights into the roles the leading eigenvalues, sample size, and dimensionality play in principal component analysis. The results are proven by a technical device, which swaps the role of rows and columns and converts the high-dimensional problems into low-dimensional one…
In this paper we propose two schemes for the recovery of the spectrum of a covariance matrix from the empirical covariance matrix, in the case where the dimension of the matrix is a subunitary multiple of the number of observations. We test, compare and analyze these on simulated data and also on some data coming from the stock market.
Spectrally-Corrected and Regularized QDA Classifier for Spiked Covariance Model
School of Data Sciences, Zhejiang University of Finance and Economics（浙江财经大学数据科学学院）；School of Mathematics and Statistics, Northeast Normal University（东北师范大学数学与统计学院）；Northeast Normal University（东北师范大学）
Covariance matrices are fundamental to the analysis and forecast of economic, physical and biological systems. Although the eigenvalues $\{λ_i\}$ and eigenvectors $\{{\bf u}_i\}$ of a covariance matrix are central to such endeavors, in practice one must inevitably approximate the covariance matrix based on data with finite sample size $n$ to obtain empirical eigenvalues $\{\tildeλ_i\}$ and eigenvectors $\{\tilde{\bf u}_i\}$, and therefore understanding the error so introduced is of central importance. We analyze eigenvector error $\|{\bf u}_i - \tilde{\bf u}_i \|^2$ while leveraging the assump…
