In this paper, we establish some new central limit theorems for certain spectral statistics of a high-dimensional sample covariance matrix under a divergent spectral norm population model. This model covers the divergent spiked population model as a special case. Meanwhile, the number of the spiked eigenvalues can either be fixed or grow to infinity. It is seen from our theorems that the divergence of population spectral norm affects the fluctuations of the linear spectral statistics in a fickle way, depending on the divergence rate.
Under the high-dimensional setting that data dimension and sample size tend to infinity proportionally, we derive the central limit theorem (CLT) for linear spectral statistics (LSS) of large-dimensional sample covariance matrix. Different from existing literature, our results do not require the assumption that the population covariance matrices are bounded. Moreover, many common kernel functions in the real data such as logarithmic functions and polynomial functions are allowed in this paper. In our model, the number of spiked eigenvalues can be fixed or tend to infinity. One salient feature …
In this paper, we establish the central limit theorem (CLT) for linear spectral statistics (LSS) of large-dimensional sample covariance matrix when the population covariance matrices are not uniformly bounded, which is a nontrivial extension of the Bai-Silverstein theorem (BST) (2004). The latter has strongly stimulated the development of high-dimensional statistics, especially the application of random matrix theory to statistics. However, the assumption of uniform boundedness of the population covariance matrices is found strongly limited to the applications of BST. The aim of this paper is …
We establish bounds for the covariance of a large class of functions of infinite variance stable random variables, including unbounded functions such as the power function and the logarithm. These bounds involve measures of dependence between the stable variables, some of which are new. The bounds are also used to deduce the central limit theorem for unbounded functions of stable moving average time series. This result extends the earlier results of Tailen Hsing and the authors on central limit theorems for bounded functions of stable moving averages. It can be used to show asymptotic normalit…
This article presents a homogeneity test for testing the equality of several high-dimensional covariance matrices for stationary processes with ignoring the assumption of normality. We give the asymptotic distribution of the proposed test. The simulation illustrates that the proposed test has perfect performance. Moreover, the power of the test can approach any high probability uniformly on a set of covariance matrices.
We propose an independence test for random variables valued into metric spaces by using a test statistic obtained from appropriately centering and rescaling the squared Hilbert-Schmidt norm of the usual empirical estimator of normalized cross-covariance operator. We then get asymptotic normality of this statistic under independence hypothesis, so leading to a new test for independence of functional random variables. A simulation study that allows to compare the proposed test to existing ones is provided.
We consider the asymptotic normality in $L^2$ of kernel estimators of the long run covariance kernel of stationary functional time series. Our results are established assuming a weakly dependent Bernoulli shift structure for the underlying observations, which contains most stationary functional time series models, under mild conditions. As a corollary, we obtain joint asymptotics for functional principal components computed from empirical long run covariance operators, showing that they have the favorable property of being asymptotically independent.
Multivariate processes with long-range dependence properties can be encountered in many fields of application. Two fundamental characteristics in such frameworks are long-range dependence parameters and correlations between component time series. We consider multivariate long-range dependent linear processes, not necessarily Gaussian. We show that the covariances between the wavelet coefficients in this setting are asymptotically Gaussian. We also study the asymptotic distributions of the estimators of the long-range dependence parameter and the long-run covariance by a wavelet-based Whittle p…
We consider estimation of the covariance matrix of a multivariate random vector under the constraint that certain covariances are zero. We first present an algorithm, which we call Iterative Conditional Fitting, for computing the maximum likelihood estimator of the constrained covariance matrix, under the assumption of multivariate normality. In contrast to previous approaches, this algorithm has guaranteed convergence properties. Dropping the assumption of multivariate normality, we show how to estimate the covariance matrix in an empirical likelihood approach. These approaches are then compa…
The asymptotic normality for a large family of eigenvalue statistics of a general sample covariance matrix is derived under the ultra-high dimensional setting, that is, when the dimension to sample size ratio $p/n \to \infty$. Based on this CLT result, we first adapt the covariance matrix test problem to the new ultra-high dimensional context. Then as a second application, we develop a new test for the separable covariance structure of a matrix-valued white noise. Simulation experiments are conducted for the investigation of finite-sample properties of the general asymptotic normality of eigen…
This paper considers testing a covariance matrix $Σ$ in the high dimensional setting where the dimension $p$ can be comparable or much larger than the sample size $n$. The problem of testing the hypothesis $H_0:Σ=Σ_0$ for a given covariance matrix $Σ_0$ is studied from a minimax point of view. We first characterize the boundary that separates the testable region from the non-testable region by the Frobenius norm when the ratio between the dimension $p$ over the sample size $n$ is bounded. A test based on a $U$-statistic is introduced and is shown to be rate optimal over this asymptotic regime.…
Let $\mathbf{A}=\frac{1}{\sqrt{np}}(\mathbf{X}^T\mathbf{X}-p\mathbf {I}_n)$ where $\mathbf{X}$ is a $p\times n$ matrix, consisting of independent and identically distributed (i.i.d.) real random variables $X_{ij}$ with mean zero and variance one. When $p/n\to\infty$, under fourth moment conditions a central limit theorem (CLT) for linear spectral statistics (LSS) of $\mathbf{A}$ defined by the eigenvalues is established. We also explore its applications in testing whether a population covariance matrix is an identity matrix.
Maximum likelihood estimation of Gaussian mixture models with different class-specific covariance matrices is known to be problematic. This is due to the unboundedness of the likelihood, together with the presence of spurious maximizers. Existing methods to bypass this obstacle are based on the fact that unboundedness is avoided if the eigenvalues of the covariance matrices are bounded away from zero. This can be done imposing some constraints on the covariance matrices, i.e. by incorporating a priori information on the covariance structure of the mixture components. The present work introduce…
A joint limit theorem for the point process of the off-diagonal entries of a sample covariance matrix $\mathbf{S}$, constructed from $n$ observations of a $p$-dimensional random vector with iid components, and the Frobenius norm of $\mathbf{S}$ is proved. In particular, assuming that $p$ and $n$ tend to infinity we obtain a central limit theorem for the Frobenius norm in the case of finite fourth moment of the components and an infinite variance stable law in the case of infinite fourth moment. Extending a theorem of Kallenberg, we establish asymptotic independence of the point process and the…
Estimation of the mean vector and covariance matrix is of central importance in the analysis of multivariate data. In the framework of generalized linear models, usually the variances are certain functions of the means with the normal distribution being an exception. We study some implications of functional relationships between covariance and the mean by focusing on the maximum likelihood and Bayesian estimation of the mean-covariance under the joint constraint $\bmΣ\bmμ = \bmμ$ for a multivariate normal distribution. A novel structured covariance is proposed through reparameterization of the…
In this paper, we propose a new test for the equality of several covariance functions for functional data. Its test statistic is taken as the supremum value of the sum of the squared differences between the estimated individual covariance functions and the pooled sample covariance function, hoping to obtain a more powerful test than some existing tests for the same testing problem. The asymptotic random expression of this test statistic under the null hypothesis is obtained. To approximate the null distribution of the proposed test statistic, we describe a parametric bootstrap method and a non…
An admissible estimator of the eigenvalues of the variance-covariance matrix is given for multivariate normal distributions with respect to the scale-invariant squared error loss.
In the estimation of the mean matrix in a multivariate normal distribution, the generalized Bayes estimators with closed forms are provided, and the sufficient conditions for their minimaxity are derived relative to both matrix and scalar quadratic loss functions. The generalized Bayes estimators of the covariance matrix are also given with closed forms, and the dominance properties are discussed for the Stein loss function.
Let $X$ be a centered random vector taking values in $\mathbb{R}^d$ and let $Σ= \mathbb{E}(X\otimes X)$ be its covariance matrix. We show that if $X$ satisfies an $L_4-L_2$ norm equivalence, there is a covariance estimator $\hatΣ$ that exhibits the optimal performance one would expect had $X$ been a gaussian vector. The procedure also improves the current state-of-the-art regarding high probability bounds in the subgaussian case (sharp results were only known in expectation or with constant probability). In both scenarios the new bound does not depend explicitly on the dimension $d$, but rathe…
We consider an inference on the eigenvalues of the covariance matrix of a multivariate normal distribution. The family of multivariate normal distributions with a fixed mean is seen as a Riemannian manifold with Fisher information metric. Two submanifolds naturally arises; one is the submanifold given by fixed eigenvectors of the covariance matrix, the other is the one given by fixed eigenvalues. We analyze the geometrical structures of these manifolds such as metric, embedding curvature under $e$-connection or $m$-connection. Based on these results, we study 1) the bias of the sample eigenval…
In this paper we propose statistical inference tools for the covariance operators of functional time series in the two sample and change point problem. In contrast to most of the literature the focus of our approach is not testing the null hypothesis of exact equality of the covariance operators. Instead we propose to formulate the null hypotheses in them form that "the distance between the operators is small", where we measure deviations by the sup-norm. We provide powerful bootstrap tests for these type of hypotheses, investigate their asymptotic properties and study their finite sample prop…
The joint distribution of two off-diagonal Wishart matrix elements was useful in recent work on geometric probability [Finch 2010]. Not finding such formulas in the literature, we report these here.
We show that in a common high-dimensional covariance model, the choice of loss function has a profound effect on optimal estimation. In an asymptotic framework based on the Spiked Covariance model and use of orthogonally invariant estimators, we show that optimal estimation of the population covariance matrix boils down to design of an optimal shrinker $η$ that acts elementwise on the sample eigenvalues. Indeed, to each loss function there corresponds a unique admissible eigenvalue shrinker $η^*$ dominating all other shrinkers. The shape of the optimal shrinker is determined by the choice of l…
Estimating covariance matrices is a problem of fundamental importance in multivariate statistics. In practice it is increasingly frequent to work with data matrices $X$ of dimension $n\times p$, where $p$ and $n$ are both large. Results from random matrix theory show very clearly that in this setting, standard estimators like the sample covariance matrix perform in general very poorly. In this "large $n$, large $p$" setting, it is sometimes the case that practitioners are willing to assume that many elements of the population covariance matrix are equal to 0, and hence this matrix is sparse. W…
We obtain the optimal Bayesian minimax rate for the unconstrained large covariance matrix of multivariate normal sample with mean zero, when both the sample size, n, and the dimension, p, of the covariance matrix tend to infinity. Traditionally the posterior convergence rate is used to compare the frequentist asymptotic performance of priors, but defining the optimality with it is elusive. We propose a new decision theoretic framework for prior selection and define Bayesian minimax rate. Under the proposed framework, we obtain the optimal Bayesian minimax rate for the spectral norm for all rat…
Covariance matrix plays a central role in multivariate statistical analysis. Significant advances have been made recently on developing both theory and methodology for estimating large covariance matrices. However, a minimax theory has yet been developed. In this paper we establish the optimal rates of convergence for estimating the covariance matrix under both the operator norm and Frobenius norm. It is shown that optimal procedures under the two norms are different and consequently matrix estimation under the operator norm is fundamentally different from vector estimation. The minimax upper …
This paper considers sparse spiked covariance matrix models in the high-dimensional setting and studies the minimax estimation of the covariance matrix and the principal subspace as well as the minimax rank detection. The optimal rate of convergence for estimating the spiked covariance matrix under the spectral norm is established, which requires significantly different techniques from those for estimating other structured covariance matrices such as bandable or sparse covariance matrices. We also establish the minimax rate under the spectral norm for estimating the principal subspace, the pri…
Consider the $p\times p$ matrix that is the product of a population covariance matrix and the inverse of another population covariance matrix. Suppose that their difference has a divergent rank with respect to $p$, when two samples of sizes $n$ and $T$ from the two populations are available, we construct its corresponding sample version. In the regime of high dimension where both $n$ and $T$ are proportional to $p$, we investigate the limiting laws for extreme (spiked) eigenvalues of the sample (spiked) Fisher matrix when the number of spikes is divergent and these spikes are unbounded.
We develop a method for estimating well-conditioned and sparse covariance and inverse covariance matrices from a sample of vectors drawn from a sub-gaussian distribution in high dimensional setting. The proposed estimators are obtained by minimizing the quadratic loss function and joint penalty of `1 norm and variance of its eigenvalues. In contrast to some of the existing methods of covariance and inverse covariance matrix estimation, where often the interest is to estimate a sparse matrix, the proposed method is flexible in estimating both a sparse and well-conditioned covariance matrix simu…
In this paper, we will discuss the concept of an array variate random variable and introduce a class of skew normal array densities that are obtained through a selection model that uses the array variate normal density as the kernel and the cumulative distribution of the univariate normal distribution as the selection function.
This paper develops mixed-normal approximations for probabilities that vectors of multiple Skorohod integrals belong to random convex polytopes when the dimensions of the vectors possibly diverge to infinity. We apply the developed theory to establish the asymptotic mixed normality of the realized covariance matrix of a high-dimensional continuous semimartingale observed at a high-frequency, where the dimension can be much larger than the sample size. We also present an application of this result to testing the residual sparsity of a high-dimensional continuous-time factor model.
Gaussian covariance graph model is a popular model in revealing underlying dependency structures among random variables. A Bayesian approach to the estimation of covariance structures uses priors that force zeros on some off-diagonal entries of covariance matrices and put a positive definite constraint on matrices. In this paper, we consider a spike and slab prior on off-diagonal entries, which uses a mixture of point-mass and normal distribution. The point-mass naturally introduces sparsity to covariance structures so that the resulting posterior from this prior renders covariance structure l…
For the multi-sample equal covariance function (ECF) testing problem, Zhang (2013) proposed an $L^{2}$-norm based test. However, its asymptotic power and finite sample performance have not been studied. In this paper, its asymptotic power is investigated under some mild conditions. It is shown that the $L^2$-norm based test is root-$n$ consistent. In addition, intensive simulation studies demonstrate that in terms of size-controlling and power, the $L^{2}$-norm based test outperforms the dimension-reduction based test proposed by Fremdt et al. (2013) when the functional data are less correlate…
In this paper we propose a perturbative method for the reconstruction of the covariance matrix of a multinormal distribution, under the assumption that the only available information amounts to the covariance matrix of a spherically truncated counterpart of the same distribution. We expand the relevant equations up to the fourth perturbative order and discuss the analytic properties of the first few perturbative terms. We finally compare the proposed approach with an exact iterative algorithm (presented in Palombi et al. (2017)) in the hypothesis that the spherically truncated covariance matri…
We propose a Bayesian methodology for estimating spiked covariance matrices with jointly sparse structure in high dimensions. The spiked covariance matrix is reparametrized in terms of the latent factor model, where the loading matrix is equipped with a novel matrix spike-and-slab LASSO prior, which is a continuous shrinkage prior for modeling jointly sparse matrices. We establish the rate-optimal posterior contraction for the covariance matrix with respect to the operator norm as well as that for the principal subspace with respect to the projection operator norm loss. We also study the poste…
Riemannian Gaussian distributions were initially introduced as basic building blocks for learning models which aim to capture the intrinsic structure of statistical populations of positive-definite matrices (here called covariance matrices). While the potential applications of such models have attracted significant attention, a major obstacle still stands in the way of these applications: there seems to exist no practical method of computing the normalising factors associated with Riemannian Gaussian distributions on spaces of high-dimensional covariance matrices. The present paper shows that …
Although the operator (spectral) norm is one of the most widely used metrics for covariance estimation, comparatively little is known about the fluctuations of error in this norm. To be specific, let $\hatΣ$ denote the sample covariance matrix of $n$ observations in $\mathbb{R}^p$ that arise from a population matrix $Σ$, and let $T_n=\sqrt{n}\|\hatΣ-Σ\|_{\text{op}}$. In the setting where the eigenvalues of $Σ$ have a decay profile of the form $λ_j(Σ)\asymp j^{-2β}$, we analyze how well the bootstrap can approximate the distribution of $T_n$. Our main result shows that up to factors of $\log(n)…
This paper aims at achieving a simultaneously sparse and low-rank estimator from the semidefinite population covariance matrices. We first benefit from a convex optimization which develops $l_1$-norm penalty to encourage the sparsity and nuclear norm to favor the low-rank property. For the proposed estimator, we then prove that with large probability, the Frobenious norm of the estimation rate can be of order $O(\sqrt{s(\log{r})/n})$ under a mild case, where $s$ and $r$ denote the number of sparse entries and the rank of the population covariance respectively, $n$ notes the sample capacity. Fi…
