We consider the eigenvalues of sample covariance matrices of the form $\mathcal{Q}=(Σ^{1/2}X)(Σ^{1/2}X)^*$. The sample $X$ is an $M\times N$ rectangular random matrix with real independent entries and the population covariance matrix $Σ$ is a positive definite diagonal matrix independent of $X$. Assuming that the limiting spectral density of $Σ$ exhibits convex decay at the right edge of the spectrum, in the limit $M, N \to \infty$ with $N/M \to d\in(0,\infty)$, we find a certain threshold $d_+$ such that for $d>d_+$ the limiting spectral distribution of $\mathcal{Q}$ also exhibits convex decay at the right edge of the spectrum. In this case, the largest eigenvalues of $\mathcal{Q}$ are determined by the order statistics of the eigenvalues of $Σ$, and in particular, the limiting distribution of the largest eigenvalue of $\mathcal{Q}$ is given by a Weibull distribution. In case $d<d_+$, we also prove that the limiting distribution of the largest eigenvalue of $\caQ$ is Gaussian if the entries of $Σ$ are i.i.d. random variables. While $Σ$ is considered to be random mostly, the results also hold for deterministic $Σ$ with some additional assumptions.
This article provides a central limit theorem for a consistent estimator of population eigenvalues with large multiplicities based on sample covariance matrices. The focus is on limited sample size situations, whereby the number of available observations is known and comparable in magnitude to the observation dimension. An exact expression as well as an empirical, asymptotically accurate, approximation of the limiting variance is derived. Simulations are performed that corroborate the theoretical claims. A specific application to wireless sensor networks is developed.
We study the asymptotic distributions of the spiked eigenvalues and the largest nonspiked eigenvalue of the sample covariance matrix under a general covariance matrix model with divergent spiked eigenvalues, while the other eigenvalues are bounded but otherwise arbitrary. The limiting normal distribution for the spiked sample eigenvalues is established. It has distinct features that the asymptotic mean relies on not only the population spikes but also the nonspikes and that the asymptotic variance in general depends on the population eigenvectors. In addition, the limiting Tracy-Widom law for …
We consider general high-dimensional spiked sample covariance models and show that their leading sample spiked eigenvalues and their linear spectral statistics are asymptotically independent when the sample size and dimension are proportional to each other. As a byproduct, we also establish the central limit theorem of the leading sample spiked eigenvalues by removing the block diagonal assumption on the population covariance matrix, which is commonly needed in the literature. Moreover, we propose consistent estimators of the $L_4$ norm of the spiked population eigenvectors. Based on these res…
The asymptotic normality for a large family of eigenvalue statistics of a general sample covariance matrix is derived under the ultra-high dimensional setting, that is, when the dimension to sample size ratio $p/n \to \infty$. Based on this CLT result, we first adapt the covariance matrix test problem to the new ultra-high dimensional context. Then as a second application, we develop a new test for the separable covariance structure of a matrix-valued white noise. Simulation experiments are conducted for the investigation of finite-sample properties of the general asymptotic normality of eigen…
There has been a growing interest in providing models for multivariate spatial processes. A majority of these models specify a parametric matrix covariance function. Based on observations, the parameters are estimated by maximum likelihood or variants thereof. While the asymptotic properties of maximum likelihood estimators for univariate spatial processes have been analyzed in detail, maximum likelihood estimators for multivariate spatial processes have not received their deserved attention yet. In this article we consider the classical increasing-domain asymptotic setting restricting the min…
We consider a more generalized spiked covariance matrix $Σ$, which is a general non-definite matrix with the spiked eigenvalues scattered into a few bulks and the largest ones allowed to tend to infinity. By relaxing the matching of the 4th moment to a tail probability decay, a {\it Generalized Four Moment Theorem} (G4MT) is proposed to show the universality of the asymptotic law for the local spectral statistics of generalized spiked covariance matrices, which implies the limiting distribution of the spiked eigenvalues of the generalized spiked covariance matrix is independent of the actual d…
This paper investigates limiting properties of eigenvalues of multivariate sample spatial-sign covariance matrices when both the number of variables and the sample size grow to infinity. The underlying p-variate populations are general enough to include the popular independent components model and the family of elliptical distributions. A first result of the paper establishes that the distribution of the eigenvalues converges to a deterministic limit that belongs to the family of generalized Marcenko-Pastur distributions. Furthermore, a new central limit theorem is established for a class of l…
Let $\mathbf{X}_n=(x_{ij})$ be a $k \times n$ data matrix with complex-valued, independent and standardized entries satisfying a Lindeberg-type moment condition. We consider simultaneously $R$ sample covariance matrices $\mathbf{B}_{nr}=\frac1n \mathbf{Q}_r \mathbf{X}_n \mathbf{X}_n^*\mathbf{Q}_r^\top,~1\le r\le R$, where the $\mathbf{Q}_{r}$'s are nonrandom real matrices with common dimensions $p\times k~(k\geq p)$. Assuming that both the dimension $p$ and the sample size $n$ grow to infinity, the limiting distributions of the eigenvalues of the matrices $\{\mathbf{B}_{nr}\}$ are identified, …
In this work, we are motivated by the recent work of Zhang et al. (2019) and study a new invariant test for equality of two large scale covariance matrices. Two modified likelihood ratio tests (LRTs) by Zhang et al. (2019) are based on the sum of log of eigenvalues (or 1- eigenvalues) of the Beta-matrix. However, as the dimension increases, many eigenvalues of the Beta-matrix are close to 0 or 1 and the modified LRTs are greatly influenced by them. In this work, instead, we consider the simple sum of the eigenvalues (of the Beta-matrix) and compute its asymptotic normality when all $n_1, n_2, …
We study the problem of detecting an abrupt change to the signal covariance matrix. In particular, the covariance changes from a "white" identity matrix to an unknown spiked or low-rank matrix. Two sequential change-point detection procedures are presented, based on the largest and the smallest eigenvalues of the sample covariance matrix. To control false-alarm-rate, we present an accurate theoretical approximation to the average-run-length (ARL) and expected detection delay (EDD) of the detection, leveraging the extreme eigenvalue distributions from random matrix theory and by capturing a non…
The spiked covariance model has gained increasing popularity in high-dimensional data analysis. A fundamental problem is determination of the number of spiked eigenvalues, $K$. For estimation of $K$, most attention has focused on the use of $top$ eigenvalues of sample covariance matrix, and there is little investigation into proper ways of utilizing $bulk$ eigenvalues to estimate $K$. We propose a principled approach to incorporating bulk eigenvalues in the estimation of $K$. Our method imposes a working model on the residual covariance matrix, which is assumed to be a diagonal matrix whose en…
We consider a spiked population model, proposed by Johnstone, whose population eigenvalues are all unit except for a few fixed eigenvalues. The question is to determine how the sample eigenvalues depend on the non-unit population ones when both sample size and population size become large. This paper completely determines the almost sure limits for a general class of samples.
In this paper, we use a new approach to prove that the largest eigenvalue of the sample covariance matrix of a normally distributed vector is bigger than the true largest eigenvalue with probability 1 when the dimension is infinite. We prove a similar result for the smallest eigenvalue.
We study the $k$-largest eigenvalues of heavy-tailed sample covariance matrices of the form $\bX\bX^\T$ in an asymptotic framework, where the dimension of the data and the sample size tend to infinity. To this end, we assume that the rows of $\bX$ are given by independent copies of some stationary process with regularly varying marginals with index $α\in(0,2)$ satisfying large deviation and mixing conditions. We apply these general results to stochastic volatility and GARCH processes.
We prove the first explicit rate of convergence to the Tracy-Widom distribution for the fluctuation of the largest eigenvalue of sample covariance matrices that are not integrable. Our primary focus is matrices of type $ X^*X $ and the proof follows the Erdös-Schlein-Yau dynamical method. We use a recent approach to the analysis of the Dyson Brownian motion from [5] to obtain a quantitative error estimate for the local relaxation flow at the edge. Together with a quantitative version of the Green function comparison theorem, this gives the rate of convergence. Combined with a result of Lee-Sch…
We establish a quantitative version of the Tracy--Widom law for the largest eigenvalue of high dimensional sample covariance matrices. To be precise, we show that the fluctuations of the largest eigenvalue of a sample covariance matrix $X^*X$ converge to its Tracy--Widom limit at a rate nearly $N^{-1/3}$, where $X$ is an $M \times N$ random matrix whose entries are independent real or complex random variables, assuming that both $M$ and $N$ tend to infinity at a constant rate. This result improves the previous estimate $N^{-2/9}$ obtained by Wang [73]. Our proof relies on a Green function comp…
For sample covariance matrices with iid entries with sub-Gaussian tails, when both the number of samples and the number of variables become large and the ratio approaches to one, it is a well-known result of A. Soshnikov that the limiting distribution of the largest eigenvalue is same as the of Gaussian samples. In this paper, we extend this result to two cases. The first case is when the ratio approaches to an arbitrary finite value. The second case is when the ratio becomes infinity or arbitrarily small.
In this paper we study the joint distributional convergence of the largest eigenvalues of the sample covariance matrix of a $p$-dimensional time series with iid entries when $p$ converges to infinity together with the sample size $n$. We consider only heavy-tailed time series in the sense that the entries satisfy some regular variation condition which ensures that their fourth moment is infinite. In this case, Soshnikov [31, 32] and Auffinger et al. [2] proved the weak convergence of the point processes of the normalized eigenvalues of the sample covariance matrix towards an inhomogeneous Pois…
We consider the asymptotic fluctuation behavior of the largest eigenvalue of certain sample covariance matrices in the asymptotic regime where both dimensions of the corresponding data matrix go to infinity. More precisely, let $X$ be an $n\times p$ matrix, and let its rows be i.i.d. complex normal vectors with mean 0 and covariance $Σ_p$. We show that for a large class of covariance matrices $Σ_p$, the largest eigenvalue of $X^*X$ is asymptotically distributed (after recentering and rescaling) as the Tracy--Widom distribution that appears in the study of the Gaussian unitary ensemble. We give…
We consider sample covariance matrices of the form $\mathcal{Q}=(Σ^{1/2}X)(Σ^{1/2} X)^*$, where the sample $X$ is an $M\times N$ random matrix whose entries are real independent random variables with variance $1/N$ and where $Σ$ is an $M\times M$ positive-definite deterministic matrix. We analyze the asymptotic fluctuations of the largest rescaled eigenvalue of $\mathcal{Q}$ when both $M$ and $N$ tend to infinity with $N/M\to d\in(0,\infty)$. For a large class of populations $Σ$ in the sub-critical regime, we show that the distribution of the largest rescaled eigenvalue of $\mathcal{Q}$ is giv…
We introduce a method for the comparison of some extremal eigenvalue statistics of random matrices. For example, it allows one to compare the maximal eigenvalue gap in the bulk of two generalized Wigner ensembles, provided that the first four moments of their matrix entries match. As an application, we extend results of Bourgade--Ben Arous and Feng--Wei that identify the limit of the maximal eigenvalue gap in the bulk of the GUE to all complex Hermitian generalized Wigner matrices.
We derive the distribution of the eigenvalues of a large sample covariance matrix when the data is dependent in time. More precisely, the dependence for each variable $i=1,...,p$ is modelled as a linear process $(X_{i,t})_{t=1,...,n}=(\sum_{j=0}^\infty c_j Z_{i,t-j})_{t=1,...,n}$, where $\{Z_{i,t}\}$ are assumed to be independent random variables with finite fourth moments. If the sample size $n$ and the number of variables $p=p_n$ both converge to infinity such that $y=\lim_{n\to\infty}{n/p_n}>0$, then the empirical spectral distribution of $p^{-1}\X\X^T$ converges to a non\hyp{}random distri…
In this paper, we study the asymptotic behavior of the extreme eigenvalues and eigenvectors of the spiked covariance matrices, in the supercritical regime. Specifically, we derive the joint distribution of the extreme eigenvalues and the generalized components of their associated eigenvectors in this regime.
In this paper we consider the extreme behavior of the extremal eigenvalues of white Wishart matrices, which plays an important role in multivariate analysis. In particular, we focus on the case when the dimension of the feature p is much larger than or comparable to the number of observations n, a common situation in modern data analysis. We provide asymptotic approximations and bounds for the tail probabilities of the extremal eigenvalues. Moreover, we construct efficient Monte Carlo simulation algorithms to compute the tail probabilities. Simulation results show that our method has the best …
Let $ \bbB_n =\frac{1}{n}(\bbR_n + \bbT^{1/2}_n \bbX_n)(\bbR_n + \bbT^{1/2}_n \bbX_n)^* $, where $ \bbX_n $ is a $ p \times n $ matrix with independent standardized random variables, $ \bbR_n $ is a $ p \times n $ non-random matrix and $ \bbT_{n} $ is a $ p \times p $ non-random, nonnegative definite Hermitian matrix. The matrix $\bbB_n$ is referred to as the information-plus-noise type matrix, where $\bbR_n$ contains the information and $\bbT^{1/2}_n \bbX_n$ is the noise matrix with the covariance matrix $\bbT_{n} $. It is known that, as $ n \to \infty $, if $ p/n $ converges to a positive nu…
We study the joint limit distribution of the $k$ largest eigenvalues of a $p\times p$ sample covariance matrix $XX^\T$ based on a large $p\times n$ matrix $X$. The rows of $X$ are given by independent copies of a linear process, $X_{it}=\sum_j c_j Z_{i,t-j}$, with regularly varying noise $(Z_{it})$ with tail index $α\in(0,4)$. It is shown that a point process based on the eigenvalues of $XX^\T$ converges, as $n\to\infty$ and $p\to\infty$ at a suitable rate, in distribution to a Poisson point process with an intensity measure depending on $α$ and $\sum c_j^2$. This result is extended to random …
We derive tight lower bounds on the smallest eigenvalue of a sample covariance matrix of a centred isotropic random vector under weak or no assumptions on its components.
Consider a sample of a centered random vector with unit covariance matrix. We show that under certain regularity assumptions, and up to a natural scaling, the smallest and the largest eigenvalues of the empirical covariance matrix converge, when the dimension and the sample size both tend to infinity, to the left and right edges of the Marchenko--Pastur distribution. The assumptions are related to tails of norms of orthogonal projections. They cover isotropic log-concave random vectors as well as random vectors with i.i.d. coordinates with almost optimal moment conditions. The method is a refi…
We compute the limiting distributions of the largest eigenvalue of a complex Gaussian sample covariance matrix when both the number of samples and the number of variables in each sample become large. When all but finitely many, say $r$, eigenvalues of the covariance matrix are the same, the dependence of the limiting distribution of the largest eigenvalue of the sample covariance matrix on those distinguished $r$ eigenvalues of the covariance matrix is completely characterized in terms of an infinite sequence of new distribution functions that generalize the Tracy-Widom distributions of the ra…
We investigate the asymptotics of eigenvalues of sample covariance matrices associated with a class of non-independent Gaussian processes (separable and temporally stationary) under the Kolmogorov asymptotic regime. The limiting spectral distribution (LSD) is shown to depend explicitly on the Kolmogorov constant (a fixed limiting ratio of the sample size to the dimensionality) and parameters representing the spatio- and temporal- correlations. The Cauchy, M- and N-transforms from free harmonic analysis play key roles to this LSD calculation problem. The free multiplication law of free random v…
In this article, we establish a limiting distribution for eigenvalues of a class of auto-covariance matrices. The same distribution has been found in the literature for a regularized version of these auto-covariance matrices. The original non-regularized auto-covariance matrices are non invertible which introduce supplementary diffculties for the study of their eigenvalues through Girko's Hermitization scheme. The key result in this paper is a new polynomial lower bound for the least singular value of the resolvent matrices associated to a rank-defective quadratic function of a random matrix w…
We consider large complex random sample covariance matrices obtained from "spiked populations", that is when the true covariance matrix is diagonal with all but finitely many eigenvalues equal to one. We investigate the limiting behavior of the largest eigenvalues when the population and the sample sizes both become large. Under some conditions on moments of the sample distribution, we prove that the asymptotic fluctuations of the largest eigenvalues are the same as for a complex Gaussian sample with the same true covariance. The real setting is also considered.
We consider the GUE minor process, where a sequence of GUE matrices is drawn from the corner of a doubly infinite array of i.i.d. standard normal variables subject to the symmetry constraint. From each matrix, we take its largest eigenvalue, appropriately rescaled to converge to the standard Tracy-Widom distribution. We show the analogue of the law of iterated logarithm for this sequence, i.e. we divide the normalized n-th eigenvalue by a logarithmic factor and show the limsup of this sequence is a constant almost surely. We also give almost sure bounds for the appropriately scaled liminf.
This work is concerned with finite range bounds on the variance of individual eigenvalues of random covariance matrices, both in the bulk and at the edge of the spectrum. In a preceding paper, the author established analogous results for Wigner matrices and stated the results for covariance matrices. They are proved in the present paper. Relying on the LUE example, which needs to be investigated first, the main bounds are extended to complex covariance matrices by means of the Tao, Vu and Wang Four Moment Theorem and recent localization results by Pillai and Yin. The case of real covariance ma…
We study principal components analyses in multivariate random and mixed effects linear models, assuming a spherical-plus-spikes structure for the covariance matrix of each random effect. We characterize the behavior of outlier sample eigenvalues and eigenvectors of MANOVA variance components estimators in such models under a high-dimensional asymptotic regime. Our results show that an aliasing phenomenon may occur in high dimensions, in which eigenvalues and eigenvectors of the MANOVA estimate for one variance component may be influenced by the other components. We propose an alternative proce…
After proper rescaling and under some technical assumptions, the smallest eigenvalue of a sample covariance matrix with aspect ratio bounded away from 1 converges to the Tracy--Widom distribution. This complements the results on the largest eigenvalue, due to Soshnikov and Peche.
We consider Schrödinger operator with random decaying potential on $\ell^2 ({\bf Z}^d)$ and showed that, (i) IDS coincides with that of free Laplacian in general cases, and (ii) the set of extremal eigenvalues, after rescaling, converges to a inhomogeneous Poisson process, under certain condition on the single-site distribution, and (iii) there are "border-line" cases, such that we have Poisson statistics in the sense of (ii) above if the potential does not decay, while we do not if the potential does decay.
We present an analytical technique to compute the probability of rare events in which the largest eigenvalue of a random matrix is atypically large (i.e.\ the right tail of its large deviations). The results also transfer to the left tail of the large deviations of the smallest eigenvalue. The technique improves upon past methods by not requiring the explicit law of the eigenvalues, and we apply it to a large class of random matrices that were previously out of reach. In particular, we solve an open problem related to the performance of principal components analysis on highly correlated data, …
Random sampling of large Markov matrices with a tunable spectral gap, a nonuniform stationary distribution, and a nondegenerate limiting empirical spectral distribution (ESD) is useful. Fix $c>0$ and $p>0$. Let $A_n$ be the adjacency matrix of a random graph following $\mathrm{G}(n, p/n)$, known as the Erdős-Rényi distribution. Add $c/n$ to each entry of $A_n$ and then normalize its rows. It is shown that the resulting Markov matrix has the desired properties. Its ESD weakly converges in probability to a symmetric nondegenerate distribution, and its extremal eigenvalues, other than 1, fall in …
We consider the statistics of the extreme eigenvalues of sparse random matrices, a class of random matrices that includes the normalized adjacency matrices of the Erdős-Rényi graph $G(N,p)$. Tracy-Widom fluctuations of the extreme eigenvalues for $p\gg N^{-2/3}$ was proved in [17,46]. We prove that there is a crossover in the behavior of the extreme eigenvalues at $p\sim N^{-2/3}$. In the case that $N^{-7/9}\ll p\ll N^{-2/3}$, we prove that the extreme eigenvalues have asymptotically Gaussian fluctuations. Under a mean zero condition and when $p=CN^{-2/3}$, we find that the fluctuations of the…
This note presents some central limit theorems for the eigenvalue counting function of Wigner matrices in the form of suitable translations of results by Gustavsson and O'Rourke on the limiting behavior of eigenvalues inside the bulk of the semicircle law for Gaussian matrices. The theorems are then extended to large families of Wigner matrices by the Tao and Vu Four Moment Theorem. Similar results are developed for covariance matrices.
We study the universality of the eigenvalue statistics of the covariance matrices $\frac{1}{n}M^* M$ where $M$ is a large $p\times n$ matrix obeying condition $\bf{C1}$. In particular, as an application, we prove a variant of universality results regarding the smallest singular value of $M_{p,n}$. This paper is an extension of the results in \cite{tvcovariance} from the bulk of the spectrum up to the edge.
We prove a central limit theorem for the difference of linear eigenvalue statistics of a sample covariance matrix $\widetilde{W}$ and its minor $W$. We find that the fluctuation of this difference is much smaller than those of the individual linear statistics, as a consequence of the strong correlation between the eigenvalues of $\widetilde{W}$ and $W$. Our result identifies the fluctuation of the spatial derivative of the approximate Gaussian field in the recent paper by Dumitru and Paquette. Unlike in a similar result for Wigner matrices, for sample covariance matrices the fluctuation may en…
This work is concerned with finite range bounds on the variance of individual eigenvalues of Wigner random matrices, in the bulk and at the edge of the spectrum, as well as for some intermediate eigenvalues. Relying on the GUE example, which needs to be investigated first, the main bounds are extended to families of Hermitian Wigner matrices by means of the Tao and Vu Four Moment Theorem and recent localization results by Erdös, Yau and Yin. The case of real Wigner matrices is obtained from interlacing formulas. As an application, bounds on the expected 2-Wasserstein distance between the empir…
Universality for eigenvalue algorithms on sample covariance matrices
Courant Institute of Mathematical Sciences, New York University（纽约大学库朗数学科学研究所）；Department of Mathematics, University of California, Irvine（加州大学欧文分校数学系）
We consider the eigenvalues of a large dimensional real or complex Ginibre matrix in the region of the complex plane where their real parts reach their maximum value. This maximum follows the Gumbel distribution and that these extreme eigenvalues form a Poisson point process, asymptotically as the dimension tends to infinity. In the complex case these facts have already been established by Bender \cite{MR2594353} and in the real case by Akemann and Phillips \cite{MR3192169} even for the more general elliptic ensemble with a sophisticated saddle point analysis. The purpose of this note is to gi…
We consider $N$ by $N$ deformed Wigner random matrices of the form $X_N=H_N+A_N$, where $H_N$ is a real symmetric or complex Hermitian Wigner matrix and $A_N$ is a deterministic real bounded diagonal matrix. We prove a universal Central Limit Theorem for the linear eigenvalue statistics of $X_N$ for all mesoscopic scales both in the spectral bulk and at regular edges where the global eigenvalue density vanishes as a square root. The method relies on the characteristic function method in [47], local laws for the Green function of $X_N$ in [3, 46, 51] and analytic subordination properties of the…
