We prove a central limit theorem for the difference of linear eigenvalue statistics of a sample covariance matrix $\widetilde{W}$ and its minor $W$. We find that the fluctuation of this difference is much smaller than those of the individual linear statistics, as a consequence of the strong correlation between the eigenvalues of $\widetilde{W}$ and $W$. Our result identifies the fluctuation of the spatial derivative of the approximate Gaussian field in the recent paper by Dumitru and Paquette. Unlike in a similar result for Wigner matrices, for sample covariance matrices the fluctuation may entirely vanish.
We consider large complex random sample covariance matrices obtained from "spiked populations", that is when the true covariance matrix is diagonal with all but finitely many eigenvalues equal to one. We investigate the limiting behavior of the largest eigenvalues when the population and the sample sizes both become large. Under some conditions on moments of the sample distribution, we prove that the asymptotic fluctuations of the largest eigenvalues are the same as for a complex Gaussian sample with the same true covariance. The real setting is also considered.
After proper rescaling and under some technical assumptions, the smallest eigenvalue of a sample covariance matrix with aspect ratio bounded away from 1 converges to the Tracy--Widom distribution. This complements the results on the largest eigenvalue, due to Soshnikov and Peche.
We extend the results about the fluctuations of the matrix entries of regular functions of Wigner matrices to the case of sample covariance random matrices.
Let $M_n$ be a $n \times n$ Wigner or sample covariance random matrix, and let $μ_1(M_n), μ_2(M_n), ..., μ_n(M_n)$ denote the unordered eigenvalues of $M_n$. We study the fluctuations of the partial linear eigenvalue statistics $$ \sum_{i=1}^{n-k} f(μ_i(M_n)) $$ as $n \rightarrow \infty$ for sufficiently nice test functions $f$. We consider both the case when $k$ is fixed and when $\min{k,n-k}$ tends to infinity with $n$.
We consider two classical ensembles of the random matrix theory: the Wigner matrices and sample covariance matrices, and prove Central Limit Theorem for linear eigenvalue statistics under rather weak (comparing with results known before) conditions on the number of derivatives of the test functions and also on the number of the entries moments. Moreover, we develop a universal method which allows one to obtain automatically the bounds for the variance of differentiable test functions, if there is a bound for the variance of the trace of the resolvent of random matrix. The method is applicable …
We derive tight lower bounds on the smallest eigenvalue of a sample covariance matrix of a centred isotropic random vector under weak or no assumptions on its components.
We compute the limiting distributions of the largest eigenvalue of a complex Gaussian sample covariance matrix when both the number of samples and the number of variables in each sample become large. When all but finitely many, say $r$, eigenvalues of the covariance matrix are the same, the dependence of the limiting distribution of the largest eigenvalue of the sample covariance matrix on those distinguished $r$ eigenvalues of the covariance matrix is completely characterized in terms of an infinite sequence of new distribution functions that generalize the Tracy-Widom distributions of the ra…
In this paper, we use a new approach to prove that the largest eigenvalue of the sample covariance matrix of a normally distributed vector is bigger than the true largest eigenvalue with probability 1 when the dimension is infinite. We prove a similar result for the smallest eigenvalue.
In this article, we establish a limiting distribution for eigenvalues of a class of auto-covariance matrices. The same distribution has been found in the literature for a regularized version of these auto-covariance matrices. The original non-regularized auto-covariance matrices are non invertible which introduce supplementary diffculties for the study of their eigenvalues through Girko's Hermitization scheme. The key result in this paper is a new polynomial lower bound for the least singular value of the resolvent matrices associated to a rank-defective quadratic function of a random matrix w…
This work is concerned with finite range bounds on the variance of individual eigenvalues of random covariance matrices, both in the bulk and at the edge of the spectrum. In a preceding paper, the author established analogous results for Wigner matrices and stated the results for covariance matrices. They are proved in the present paper. Relying on the LUE example, which needs to be investigated first, the main bounds are extended to complex covariance matrices by means of the Tao, Vu and Wang Four Moment Theorem and recent localization results by Pillai and Yin. The case of real covariance ma…
We present an alternative proof of asymptotic freeness of independent sample covariance matrices, when the dimension and the sample size grow at the same rate, by embedding these matrices into Wigner matrices of a larger order and using asymptotic freeness of independent Wigner and deterministic matrices.
This note presents some central limit theorems for the eigenvalue counting function of Wigner matrices in the form of suitable translations of results by Gustavsson and O'Rourke on the limiting behavior of eigenvalues inside the bulk of the semicircle law for Gaussian matrices. The theorems are then extended to large families of Wigner matrices by the Tao and Vu Four Moment Theorem. Similar results are developed for covariance matrices.
We establish a large deviation principle for the empirical spectral measure of a sample covariance matrix with sub-Gaussian entries, which extends Bordenave and Caputo's result for Wigner matrices having the same type of entries [7]. To this aim, we need to establish an asymptotic freeness result for rectangular free convolution, more precisely, we give a bound in the subordination formula for information-plus-noise matrices.
In this paper, we study the convergent limits and rates of the eigenvalues and eigenvectors for spiked sample covariance matrices whose spectrum can have multiple bulk components. Our model is an extension of Johnstone's spiked covariance matrix model. Based on our results, we can extend many statistical applications based on Johnstone's spiked covariance matrix model.
We consider a spiked population model, proposed by Johnstone, whose population eigenvalues are all unit except for a few fixed eigenvalues. The question is to determine how the sample eigenvalues depend on the non-unit population ones when both sample size and population size become large. This paper completely determines the almost sure limits for a general class of samples.
We study the universality of the eigenvalue statistics of the covariance matrices $\frac{1}{n}M^* M$ where $M$ is a large $p\times n$ matrix obeying condition $\bf{C1}$. In particular, as an application, we prove a variant of universality results regarding the smallest singular value of $M_{p,n}$. This paper is an extension of the results in \cite{tvcovariance} from the bulk of the spectrum up to the edge.
We obtain the limiting spectral distribution for large sample covariance matrices associated with random vectors having graph-dependent entries under the assumption that the interdependence among the entries grows with the sample size n. Our results are tight. In particular, they give necessary and sufficient conditions for the Marchenko-Pastur theorem for sample covariance matrices with m-dependent orthonormal elements when m = o(n).
We consider the sample covariance matrices of large data matrices which have i.i.d. complex matrix entries and which are non-square in the sense that the difference between the number of rows and the number of columns tends to infinity. We show that the second-order correlation function of the characteristic polynomial of the sample covariance matrix is asymptotically given by the sine kernel in the bulk of the spectrum and by the Airy kernel at the edge of the spectrum. Similar results are given for real sample covariance matrices.
We investigate the second-order correlation function of the characteristic polynomial of a sample covariance matrix. Starting from an explicit formula for the generating function, we re-obtain several well-known kernels from random matrix theory.
When some eigenvalues of a spiked multiplicative resp. additive deformation model of a Hermitian Wigner matrix resp. a sample covariance matrix separate from the bulk, we study how the corresponding eigenvectors project onto those of the perturbation. We point out that the inverse of the subordination function relative to the free additive resp. multiplicative convolution plays an important part in the asymptotic behavior.
We investigate the spectral distribution of large sample covariance matrices with independent columns and entries in the columns that stem from Markov chains. We characterize the limiting spectral densities by their moments. Correspondingly, the proof is based on a moment method.
This work is concerned with finite range bounds on the variance of individual eigenvalues of Wigner random matrices, in the bulk and at the edge of the spectrum, as well as for some intermediate eigenvalues. Relying on the GUE example, which needs to be investigated first, the main bounds are extended to families of Hermitian Wigner matrices by means of the Tao and Vu Four Moment Theorem and recent localization results by Erdös, Yau and Yin. The case of real Wigner matrices is obtained from interlacing formulas. As an application, bounds on the expected 2-Wasserstein distance between the empir…
This paper is the third chapter of three of the author's undergraduate thesis. In this paper, we study the convergence of local bulk statistics for linearized covariance matrices under Dyson's Brownian motion. We consider deterministic initial data $V$ approximate the Dyson Brownian motion for linearized covariance matrices by the Wigner flow. Using universality results for the Wigner flow, we deduce universality for the linearized covariance matrices. We deduce bulk universality of averaged bulk correlation functions for both biregular bipartite graphs and honest covariance matrices. We also …
We show that the empirical distribution of the eigenvalues of the sample covariance matrix of certain random vectors (not necessarily independent entries) with bounded marginal $L^{4}$ norms converges weakly to a compound free Poisson distribution.
Universality for eigenvalue algorithms on sample covariance matrices
Courant Institute of Mathematical Sciences, New York University（纽约大学库朗数学科学研究所）；Department of Mathematics, University of California, Irvine（加州大学欧文分校数学系）
We establish a large deviation theorem for the empirical spectral distribution of random covariance matrices whose entries are independent random variables with mean 0, variance 1 and having controlled forth moments. Some new properties of Laguerre polynomials are also given.
We extend to the matrix setting a recent result of Srivastava-Vershynin about estimating the covariance matrix of a random vector. The result can be in- terpreted as a quantified version of the law of large numbers for positive semi-definite matrices which verify some regularity assumption. Beside giving examples, we dis- cuss the notion of log-concave matrices and give estimates on the smallest and largest eigenvalues of a sum of such matrices.
We study the $k$-largest eigenvalues of heavy-tailed sample covariance matrices of the form $\bX\bX^\T$ in an asymptotic framework, where the dimension of the data and the sample size tend to infinity. To this end, we assume that the rows of $\bX$ are given by independent copies of some stationary process with regularly varying marginals with index $α\in(0,2)$ satisfying large deviation and mixing conditions. We apply these general results to stochastic volatility and GARCH processes.
This article provides a central limit theorem for a consistent estimator of population eigenvalues with large multiplicities based on sample covariance matrices. The focus is on limited sample size situations, whereby the number of available observations is known and comparable in magnitude to the observation dimension. An exact expression as well as an empirical, asymptotically accurate, approximation of the limiting variance is derived. Simulations are performed that corroborate the theoretical claims. A specific application to wireless sensor networks is developed.
We consider sparse sample covariance matrices $\frac1{np_n}\mathbf X\mathbf X^*$, where $\mathbf X$ is a sparse matrix of order $n\times m$ with the sparse probability $p_n$. We prove the local Marchenko--Pastur law in some complex domain assuming that $np_n>\log^βn$, $β>0$ and some $(4+δ)$-moment condition is fulfilled, $δ>0$.
We consider $N$ by $N$ deformed Wigner random matrices of the form $X_N=H_N+A_N$, where $H_N$ is a real symmetric or complex Hermitian Wigner matrix and $A_N$ is a deterministic real bounded diagonal matrix. We prove a universal Central Limit Theorem for the linear eigenvalue statistics of $X_N$ for all mesoscopic scales both in the spectral bulk and at regular edges where the global eigenvalue density vanishes as a square root. The method relies on the characteristic function method in [47], local laws for the Green function of $X_N$ in [3, 46, 51] and analytic subordination properties of the…
In this paper, we consider a data matrix $X\in\mathbb{C}^{N\times M}$ where all the columns are i.i.d. samples being $N$ dimensional complex Gaussian of mean zero and covariance $Σ\in\mathbb{C}^{N\times N}$. Here the population matrix $Σ$ is of finite rank perturbation of the identity matrix. This is the "spiked population model" first proposed by Johnstone in \cite{21}. As $M, N\to\infty$ but $M/N = γ\in(1, \infty)$, we first establish in this paper the asymptotic distribution of the smallest eigenvalue of the sample covariance matrix $S:= XX^*/M$. It also exhibits a phase transition phenomen…
A central limit theorem (CLT) for the smoothed empirical spectral distribution of sample covariance matrices is established. Moreover, the CLTs for the smoothed quantiles of Marcenko and Pastur's law have been also developed.
We establish a large-deviations principle for the largest eigenvalue of a generalized sample covariance matrix, meaning a matrix proportional to $Z^T ΓZ$, where $Z$ has i.i.d. real or complex entries and $Γ$ is not necessarily the identity. We treat the classical case when $Z$ is Gaussian and $Γ$ is positive definite, but we also cover two orthogonal extensions: Either the entries of $Z$ can instead be sharp sub-Gaussian, a class including Rademacher and uniform distributions, where we find the same rate function as for the Gaussian model; or $Γ$ can have negative eigenvalues if $Z$ remains Ga…
A consistent kernel estimator of the limiting spectral distribution of general sample covariance matrices was introduced in Jing, Pan, Shao and Zhou (2010). The central limit theorem of the kernel estimator is proved in this paper.
We derive the universality principle for empirical spectral distributions of sample covariance matrices and their Stieltjes transforms. This principle states the following. Suppose quadratic forms of random vectors $y_p$ in $R^p$ satisfy a weak law of large numbers and the sample size grows at the same rate as $p$. Then the limiting spectral distribution of corresponding sample covariance matrices is the same as in the case with conditionally Gaussian $y_p$. This result is generalized for $m$-dependent martingale difference sequences and $m$-dependent linear processes.
In this paper we analyze the covariance kernel of the Gaussian process that arises as the limit of fluctuations of linear spectral statistics for Wigner matrices with a few moments. More precisely, the process we study here corresponds to Hermitian matrices with independent entries that have $α$ moments for $2<α< 4$. We obtain a closed form $α$-dependent expression for the covariance of the limiting process resulting from fluctuations of the Stieltjes transform by explicitly integrating the known double Laplace transform integral formula obtained in the literature. We then express the covarian…
We prove the first explicit rate of convergence to the Tracy-Widom distribution for the fluctuation of the largest eigenvalue of sample covariance matrices that are not integrable. Our primary focus is matrices of type $ X^*X $ and the proof follows the Erdös-Schlein-Yau dynamical method. We use a recent approach to the analysis of the Dyson Brownian motion from [5] to obtain a quantitative error estimate for the local relaxation flow at the edge. Together with a quantitative version of the Green function comparison theorem, this gives the rate of convergence. Combined with a result of Lee-Sch…
We establish a quantitative version of the Tracy--Widom law for the largest eigenvalue of high dimensional sample covariance matrices. To be precise, we show that the fluctuations of the largest eigenvalue of a sample covariance matrix $X^*X$ converge to its Tracy--Widom limit at a rate nearly $N^{-1/3}$, where $X$ is an $M \times N$ random matrix whose entries are independent real or complex random variables, assuming that both $M$ and $N$ tend to infinity at a constant rate. This result improves the previous estimate $N^{-2/9}$ obtained by Wang [73]. Our proof relies on a Green function comp…
The present work provides an original framework for random matrix analysis based on revisiting the concentration of measure theory from a probabilistic point of view. By providing various notions of vector concentration ($q$-exponential, linear, Lipschitz, convex), a set of elementary tools is laid out that allows for the immediate extension of classical results from random matrix theory involving random concentrated vectors in place of vectors with independent entries. These findings are exemplified here in the context of sample covariance matrices but find a large range of applications in st…
Consider the sample covariance matrix $$Σ^{1/2}XX^TΣ^{1/2}$$ where $X$ is an $M\times N$ random matrix with independent entries and $Σ$ is an $M\times M$ diagonal matrix. It is known that if $Σ$ is deterministic, then the fluctuation of $$\sum_if(λ_i)$$ converges in distribution to a Gaussian distribution. Here $\{λ_i\}$ are eigenvalues of $Σ^{1/2}XX^TΣ^{1/2}$ and $f$ is a good enough test function. In this paper we consider the case that $Σ$ is random and show that the fluctuation of $$\frac{1}{\sqrt N}\sum_if(λ_i)$$ converges in distribution to a Gaussian distribution. This phenomenon implie…
We study the renormalized real sample covariance matrix $H=X^TX/\sqrt{MN}-\sqrt{M/N}$ with $N/M\rightarrow0$ as $N, M\rightarrow \infty$ in this paper. And we always assume $M=M(N)$. Here $X=[X_{jk}]_{M\times N}$ is an $M\times N$ real random matrix with i.i.d entries, and we assume $\mathbb{E}|X_{11}|^{5+δ}<\infty$ with some small positive $δ$. The Stieltjes transform $m_N(z)=N^{-1}Tr(H-z)^{-1}$ and the linear eigenvalue statistics of $H$ are considered. We mainly focus on the asymptotic expansion of $\mathbb{E}\{m_N(z)\}$ in this paper. Then for some fine test function, a central limit theor…
The purpose of this note is twofold: firstly to improve the known results on variation of extreme eigenvalues of birth and death matrices and random walk matrices; and secondly to progress towards the solution of a thirty years old open problem concerning the variation of eigenvalues of these matrices. Keywords: Birth and death matrices, random walk matrices, eigenvalues, monotonicity
For sample covariance matrices with iid entries with sub-Gaussian tails, when both the number of samples and the number of variables become large and the ratio approaches to one, it is a well-known result of A. Soshnikov that the limiting distribution of the largest eigenvalue is same as the of Gaussian samples. In this paper, we extend this result to two cases. The first case is when the ratio approaches to an arbitrary finite value. The second case is when the ratio becomes infinity or arbitrarily small.
This paper is to investigate the spectral properties of sample covariance matrices under a more general population. We consider a class of matrices of the form $\mathbf S_n=\frac1n\mathbf B_n\mathbf X_n\mathbf X_n^*\mathbf B_n^*$, where $\mathbf B_n$ is a $p\times m$ non-random matrix and $\mathbf X_n$ is an $m\times n$ matrix consisting of i.i.d standard complex entries. $p/n\to c\in (0,\infty)$ as $n\to \infty$ while $m$ can be arbitrary. We proved that under some mild assumptions, with probability 1, there will be no eigenvalues in any closed interval contained in an open interval outside t…
In this paper we establish the limit of the empirical spectral distribution of quaternion sample covariance matrices. Suppose $\mathbf X_n = ({x_{jk}^{(n)}})_{p\times n}$ is a quaternion random matrix. For each $n$, the entries $\{x_{ij}^{(n)}\}$ are independent random quaternion variables with a common mean $μ$ and variance $σ^2>0$. It is shown that the empirical spectral distribution of the quaternion sample covariance matrix $\mathbf S_n=n^{-1}\mathbf X_n\mathbf X_n^*$ converges to the M-P law as $p\to\infty$, $n\to\infty$ and $p/n\to y\in(0,+\infty)$.
Consider a sample of a centered random vector with unit covariance matrix. We show that under certain regularity assumptions, and up to a natural scaling, the smallest and the largest eigenvalues of the empirical covariance matrix converge, when the dimension and the sample size both tend to infinity, to the left and right edges of the Marchenko--Pastur distribution. The assumptions are related to tails of norms of orthogonal projections. They cover isotropic log-concave random vectors as well as random vectors with i.i.d. coordinates with almost optimal moment conditions. The method is a refi…
We consider a multivariate heavy-tailed stochastic volatility model and analyze the large-sample behavior of its sample covariance matrix. We study the limiting behavior of its entries in the infinite-variance case and derive results for the ordered eigenvalues and corresponding eigenvectors. Essentially, we consider two different cases where the tail behavior either stems from the i.i.d. innovations of the process or from its volatility sequence. In both cases, we make use of a large deviations technique for regularly varying time series to derive multivariate $α$-stable limit distributions o…
We investigate the asymptotics of eigenvalues of sample covariance matrices associated with a class of non-independent Gaussian processes (separable and temporally stationary) under the Kolmogorov asymptotic regime. The limiting spectral distribution (LSD) is shown to depend explicitly on the Kolmogorov constant (a fixed limiting ratio of the sample size to the dimensionality) and parameters representing the spatio- and temporal- correlations. The Cauchy, M- and N-transforms from free harmonic analysis play key roles to this LSD calculation problem. The free multiplication law of free random v…
We consider complex sample covariance matrices $M_N=\frac{1}{N}YY^*$ where $Y$ is a $N \times p$ random matrix with i.i.d. entries $Y_{ij}, 1\leq i\leq N, 1\leq j \leq p$ with distribution $F$. Under some regularity and decay assumption on $F$, we prove universality of some local eigenvalue statistics in the bulk of the spectrum in the limit where $N\to \infty$ and $\lim_{N \to \infty}p/N =γ$ for any real number $γ\in (0, \infty)$.
In this paper we study the joint distributional convergence of the largest eigenvalues of the sample covariance matrix of a $p$-dimensional time series with iid entries when $p$ converges to infinity together with the sample size $n$. We consider only heavy-tailed time series in the sense that the entries satisfy some regular variation condition which ensures that their fourth moment is infinite. In this case, Soshnikov [31, 32] and Auffinger et al. [2] proved the weak convergence of the point processes of the normalized eigenvalues of the sample covariance matrix towards an inhomogeneous Pois…
In this short note we provide an analytical formula for the conditional covariance matrices of the elliptically distributed random vectors, when the conditioning is based on the values of any linear combination of the marginal random variables. We show that one could introduce the univariate invariant depending solely on the conditioning set, which greatly simplifies the calculations. As an application, we show that one could define uniquely defined quantile-based sets on which conditional covariance matrices must be equal to each other if only the vector is multivariate normal. The similar re…
In this paper, we study the asymptotic behavior of the extreme eigenvalues and eigenvectors of the spiked covariance matrices, in the supercritical regime. Specifically, we derive the joint distribution of the extreme eigenvalues and the generalized components of their associated eigenvectors in this regime.
It is shown that the Kolmogorov distance between the spectral distribution function of a random covariance matrix $\frac1p XX^T$, where $X$ is a $n\times p$ matrix with independent entries and the distribution function of the Marchenko-Pastur law is of order $O(n^{-1/2})$. The bounds hold {\it uniformly} for any $p$, including $\frac pn$ equal or close to 1.
