In this paper, we prove the edge universality of largest eigenvalues for separable covariance matrices of the form $\mathcal Q :=A^{1/2}XBX^*A^{1/2}$. Here $X=(x_{ij})$ is an $n\times N$ random matrix with $x_{ij}=N^{-1/2}q_{ij}$, where $q_{ij}$ are $i.i.d.$ random variables with zero mean and unit variance, and $A$ and $B$ are respectively $n \times n$ and $N\times N$ deterministic non-negative definite symmetric (or Hermitian) matrices. We consider the high-dimensional case, i.e. ${n}/{N}\to d \in (0, \infty)$ as $N\to \infty$. Assuming $\mathbb E q_{ij}^3=0$ and some mild conditions on $A$ and $B$, we prove that the limiting distribution of the largest eigenvalue of $\mathcal Q$ coincide with that of the corresponding Gaussian ensemble (i.e. the $\mathcal Q$ with $X$ being an $i.i.d.$ Gaussian matrix) as long as we have $\lim_{s \rightarrow \infty}s^4 \mathbb{P}(\vert q_{ij} \vert \geq s)=0$, which is a sharp moment condition for edge universality. If we take $B=I$, then $\mathcal Q$ becomes the normal sample covariance matrix and the edge universality holds true without the vanishing third moment condition. So far, this is the strongest edge universality result for sample covariance matrices with correlated data (i.e. non-diagonal $A$) and heavy tails, which improves the previous results in \cite{BPZ1,LS} (assuming high moments and diagonal $A$), \cite{Anisotropic} (assuming high moments) and \cite{DY} (assuming diagonal $A$).
We study the universality of the eigenvalue statistics of the covariance matrices $\frac{1}{n}M^* M$ where $M$ is a large $p\times n$ matrix obeying condition $\bf{C1}$. In particular, as an application, we prove a variant of universality results regarding the smallest singular value of $M_{p,n}$. This paper is an extension of the results in \cite{tvcovariance} from the bulk of the spectrum up to the edge.
Suppose that $\mathbf X_n=(x_{jk})$ is $N\times n$ whose elements are independent real variables with mean zero, variance 1 and the fourth moment equal to three. The separable sample covariance matrix is defined as $\mathbf{B}_n = \frac1N\mathbf{T}_{2n}^{1/2} \mathbf{X}_n \mathbf{T}_{1n} \mathbf{X}_n' \mathbf{T}_{2n}^{1/2}$ where $\mathbf{T}_{1n}$ is a symmetric matrix and $\mathbf{T}_{2n}^{1/2}$ is a symmetric square root of the nonnegative definite symmetric matrix $\mathbf{T}_{2n}$. Its linear spectral statistics (LSS) are shown to have Gaussian limits when $n/N$ approaches a positive const…
We derive the universality principle for empirical spectral distributions of sample covariance matrices and their Stieltjes transforms. This principle states the following. Suppose quadratic forms of random vectors $y_p$ in $R^p$ satisfy a weak law of large numbers and the sample size grows at the same rate as $p$. Then the limiting spectral distribution of corresponding sample covariance matrices is the same as in the case with conditionally Gaussian $y_p$. This result is generalized for $m$-dependent martingale difference sequences and $m$-dependent linear processes.
We consider complex sample covariance matrices $M_N=\frac{1}{N}YY^*$ where $Y$ is a $N \times p$ random matrix with i.i.d. entries $Y_{ij}, 1\leq i\leq N, 1\leq j \leq p$ with distribution $F$. Under some regularity and decay assumption on $F$, we prove universality of some local eigenvalue statistics in the bulk of the spectrum in the limit where $N\to \infty$ and $\lim_{N \to \infty}p/N =γ$ for any real number $γ\in (0, \infty)$.
Local Law and Outlier Eigenvalues of Spiked Separable Covariance Matrices
Qiuzhen College, Tsinghua University（清华大学邱氏书院）；Department of Probability and Statistics, School of Mathematical Sciences, Peking University（北京大学数学科学学院概率统计系）
In this paper, we prove a necessary and sufficient condition for the edge universality of sample covariance matrices with general population. We consider sample covariance matrices of the form $\mathcal Q = TX(TX)^{*}$, where the sample $X$ is an $M_2\times N$ random matrix with $i.i.d.$ entries with mean zero and variance $N^{-1}$, and $T$ is an $M_1 \times M_2$ deterministic matrix satisfying $T^* T$ is diagonal. We study the asymptotic behavior of the largest eigenvalues of $\mathcal Q$ when $M:=\min\{M_1,M_2\}$ and $N$ tends to infinity with $\lim_{N \to \infty} {N}/{M}=d \in (0, \infty)$.…
In this paper, we prove a universality result of convergence for a bivariate random process defined by the eigenvectors of a sample covariance matrix. Let $V_n=(v_{ij})_{i \leq n,\, j\leq m}$ be a $n\times m$ random matrix, where $(n/m)\to y > 0$ as $ n \to \infty$, and let $X_n=(1/m) V_n V^{*}_n $ be the sample covariance matrix associated to $V_n \:$. Consider the spectral decomposition of $X_n$ given by $ U_n D_n U_n^{*}$, where $U_n=(u_{ij})_{n\times n}$ is an eigenmatrix of $X_n$. We prove, under some moments conditions, that the bivariate random process $<B_{s,t}^{n} = \underset{1\leq j …
A central limit theorem (CLT) for the smoothed empirical spectral distribution of sample covariance matrices is established. Moreover, the CLTs for the smoothed quantiles of Marcenko and Pastur's law have been also developed.
Consider the random matrix $Σ= D^{1/2} X \widetilde D^{1/2}$ where $D$ and $\widetilde D$ are deterministic Hermitian nonnegative matrices with respective dimensions $N \times N$ and $n \times n$, and where $X$ is a random matrix with independent and identically distributed centered elements with variance $1/n$. Assume that the dimensions $N$ and $n$ grow to infinity at the same pace, and that the spectral measures of $D$ and $\widetilde D$ converge as $N,n \to\infty$ towards two probability measures. Then it is known that the spectral measure of $ΣΣ^*$ converges towards a probability measure …
After proper rescaling and under some technical assumptions, the smallest eigenvalue of a sample covariance matrix with aspect ratio bounded away from 1 converges to the Tracy--Widom distribution. This complements the results on the largest eigenvalue, due to Soshnikov and Peche.
We introduce a class of separable sample covariance matrices of the form $\widetilde{\mathcal{Q}}_1:=\widetilde A^{1/2} X \widetilde B X^* \widetilde A^{1/2}.$ Here $\widetilde{A}$ and $\widetilde{B}$ are positive definite matrices whose spectrums consist of bulk spectrums plus several spikes, i.e. larger eigenvalues that are separated from the bulks. Conceptually, we call $\widetilde{\mathcal{Q}}_1$ a \emph{spiked separable covariance matrix model}. On the one hand, this model includes the spiked covariance matrix as a special case with $\widetilde{B}=I$. On the other hand, it allows for more…
Universality for eigenvalue algorithms on sample covariance matrices
Courant Institute of Mathematical Sciences, New York University（纽约大学库朗数学科学研究所）；Department of Mathematics, University of California, Irvine（加州大学欧文分校数学系）
We present a probabilistic analysis of two Krylov subspace methods for solving linear systems. We prove a central limit theorem for norms of the residual vectors that are produced by the conjugate gradient and MINRES algorithms when applied to a wide class of sample covariance matrices satisfying some standard moment conditions. The proof involves establishing a four moment theorem for the so-called spectral measure, implying, in particular, universality for the matrix produced by the Lanczos iteration. The central limit theorem then implies an almost-deterministic iteration count for the iter…
In this paper, we consider the separable covariance model, which plays an important role in wireless communications and spatio-temporal statistics and describes a process where the time correlation does not depend on the spatial location and the spatial correlation does not depend on time. We established a central limit theorem for linear spectral statistics of general separable sample covariance matrices in the form of $\mathbf S_n=\frac1n\mathbf T_{1n}\mathbf X_n\mathbf T_{2n}\mathbf X_n^*\mathbf T_{1n}^*$ where $\mathbf X_n=(x_{jk})$ is of $m_1\times m_2$ dimension, the entries $\{x_{jk}, j…
On Universality of Non-Separable Approximate Message Passing Algorithms
Department of Statistics and Data Science, Yale University（统计与数据科学系，耶鲁大学）；Halicioglu Data Science Institute, University of California, San Diego（Halicioglu数据科学研究所，加州大学圣地亚哥分校）
Recently, harmonic functions and frequently universal harmonic functions on a tree $T$ have been studied, taking values on a separable Fréchet space $E$ over the field $\mathbb{C}$ or $\mathbb{R}$. In the present paper, we allow the functions to take values in a vector space $E$ over a rather general field $\mathbb{F}$. The metric of the separable topological vector space $E$ is translation invariant and instead of harmonic functions we can also study more general functions defined by linear combinations with coefficients in $\mathbb{F}$. Unlike the past literature, we don't assume that $E$ is…
This paper is the third chapter of three of the author's undergraduate thesis. In this paper, we study the convergence of local bulk statistics for linearized covariance matrices under Dyson's Brownian motion. We consider deterministic initial data $V$ approximate the Dyson Brownian motion for linearized covariance matrices by the Wigner flow. Using universality results for the Wigner flow, we deduce universality for the linearized covariance matrices. We deduce bulk universality of averaged bulk correlation functions for both biregular bipartite graphs and honest covariance matrices. We also …
We investigate the asymptotics of eigenvalues of sample covariance matrices associated with a class of non-independent Gaussian processes (separable and temporally stationary) under the Kolmogorov asymptotic regime. The limiting spectral distribution (LSD) is shown to depend explicitly on the Kolmogorov constant (a fixed limiting ratio of the sample size to the dimensionality) and parameters representing the spatio- and temporal- correlations. The Cauchy, M- and N-transforms from free harmonic analysis play key roles to this LSD calculation problem. The free multiplication law of free random v…
For $N,n\in\mathbb N$, consider the sample covariance matrix $$S_N(T)=\frac{1}{N}XX^*$$ from a data set $X=C_N^{1/2}ZT_n^{1/2}$, where $Z=(Z_{i,j})$ is a $N\times n$ matrix having i.i.d. entries with mean zero and variance one, and $C_N, T_n$ are deterministic positive semi-definite Hermitian matrices of dimension $N$ and $n$, respectively. We assume that $(C_N)_N$ is bounded in spectral norm, and $T_n$ is a Toeplitz matrix with its largest eigenvalues diverging to infinity. The matrix $X$ can be viewed as a data set of an $N$-dimensional long memory stationary process having separable depende…
We prove the first explicit rate of convergence to the Tracy-Widom distribution for the fluctuation of the largest eigenvalue of sample covariance matrices that are not integrable. Our primary focus is matrices of type $ X^*X $ and the proof follows the Erdös-Schlein-Yau dynamical method. We use a recent approach to the analysis of the Dyson Brownian motion from [5] to obtain a quantitative error estimate for the local relaxation flow at the edge. Together with a quantitative version of the Green function comparison theorem, this gives the rate of convergence. Combined with a result of Lee-Sch…
For sample covariance matrices with iid entries with sub-Gaussian tails, when both the number of samples and the number of variables become large and the ratio approaches to one, it is a well-known result of A. Soshnikov that the limiting distribution of the largest eigenvalue is same as the of Gaussian samples. In this paper, we extend this result to two cases. The first case is when the ratio approaches to an arbitrary finite value. The second case is when the ratio becomes infinity or arbitrarily small.
In this paper we prove the universality of covariance matrices of the form $H_{N\times N}={X}^{\dagger}X$ where $X$ is an ${M\times N}$ rectangular matrix with independent real valued entries $x_{ij}$ satisfying $\mathbb{E}x_{ij}=0$ and $\mathbb{E}x^2_{ij}={\frac{1}{M}}$, $N$, $M\to \infty$. Furthermore it is assumed that these entries have sub-exponential tails or sufficiently high number of moments. We will study the asymptotics in the regime $N/M=d_N\in(0,\infty),\lim_{N\to\infty}d_N\neq0,\infty$. Our main result is the edge universality of the sample covariance matrix at both edges of the …
Recently Johansson and Johnstone proved that the distribution of the (properly rescaled) largest principal component of the complex (real) Wishart matrix $ X^* \* X (X^t \*X) $ converges to the Tracy-Widom law as $ n, p $ (the dimensions of $ X $) tend to $ \infty $ in some ratio $ n/p \to γ>0. $ We extend these results in two directions. First of all, we prove that the joint distribution of the first, second, third, etc. eigenvalues of a Wishart matrix converges (after a proper rescaling) to the Tracy-Widom distribution. Second of all, we explain how the combinatorial machinery developed for …
In order to have a better understanding of finite random matrices with non-Gaussian entries, we study the $1/N$ expansion of local eigenvalue statistics in both the bulk and at the hard edge of the spectrum of random matrices. This gives valuable information about the smallest singular value not seen in universality laws. In particular, we show the dependence on the fourth moment (or the kurtosis) of the entries. This work makes use of the so-called complex Gaussian divisible ensembles for both Wigner and sample covariance matrices.
We are concerned with the behavior of the eigenvalues of renormalized sample covariance matrices of the form C_n=\sqrt{\frac{n}{p}}\left(\frac{1}{n}A_{p}^{1/2}X_{n}B_{n}X_{n}^{*}A_{p}^{1/2}-\frac{1}{n}\tr(B_{n})A_{p}\right) as $p,n\to \infty$ and $p/n\to 0$, where $X_{n}$ is a $p\times n$ matrix with i.i.d. real or complex valued entries $X_{ij}$ satisfying $E(X_{ij})=0$, $E|X_{ij}|^2=1$ and having finite fourth moment. $A_{p}^{1/2}$ is a square-root of the nonnegative definite Hermitian matrix $A_{p}$, and $B_{n}$ is an $n\times n$ nonnegative definite Hermitian matrix. We show that the empir…
Let $X,X_1,\dots, X_n,\dots$ be i.i.d. centered Gaussian random variables in a separable Banach space $E$ with covariance operator $Σ:$ $$ Σ:E^{\ast}\mapsto E,\ \ Σu = {\mathbb E}\langle X,u\rangle, u\in E^{\ast}. $$ The sample covariance operator $\hat Σ:E^{\ast}\mapsto E$ is defined as $$ \hat Σu := n^{-1}\sum_{j=1}^n \langle X_j,u\rangle X_j, u\in E^{\ast}. $$ The goal of the paper is to obtain concentration inequalities and expectation bounds for the operator norm $\|\hat Σ-Σ\|$ of the deviation of the sample covariance operator from the true covariance operator. In particular, it is shown…
This paper is aimed at deriving the universality of the largest eigenvalue of a class of high-dimensional real or complex sample covariance matrices of the form $\mathcal{W}_N=Σ^{1/2}XX^*Σ^{1/2}$. Here, $X=(x_{ij})_{M,N}$ is an $M\times N$ random matrix with independent entries $x_{ij},1\leq i\leq M,1\leq j\leq N$ such that $\mathbb{E}x_{ij}=0$, $\mathbb{E}|x_{ij}|^2=1/N$. On dimensionality, we assume that $M=M(N)$ and $N/M\rightarrow d\in(0,\infty)$ as $N\rightarrow\infty$. For a class of general deterministic positive-definite $M\times M$ matrices $Σ$, under some additional assumptions on th…
We consider time correlation for KPZ growth in 1+1 dimensions in a neighborhood of a characteristics. We prove convergence of the covariance with droplet, flat and stationary initial profile. In particular, this provides a rigorous proof of the exact formula of the covariance for the stationary case obtained in [SIGMA 12 (2016), 074]. Furthermore, we prove the universality of the first order correction when the two observation times are close and provide a rigorous bound of the error term. This result holds also for random initial profiles which are not necessarily stationary.
We consider the conjugate gradient algorithm applied to a general class of spiked sample covariance matrices. The main result of the paper is that the norms of the error and residual vectors at any finite step concentrate on deterministic values determined by orthogonal polynomials with respect to a deformed Marchenko--Pastur law. The first-order limits and fluctuations are shown to be universal. Additionally, for the case where the bulk eigenvalues lie in a single interval we show a stronger universality result in that the asymptotic rate of convergence of the conjugate gradient algorithm onl…
We consider an $N$ by $N$ real symmetric random matrix $X=(x_{ij})$ where $\mathbb{E}x_{ij}x_{kl}=ξ_{ijkl}$. Under the assumption that $(ξ_{ijkl})$ is the discretization of a piecewise Lipschitz function and that the correlation is short-ranged we prove that the empirical spectral measure of $X$ converges to a probability measure. The Stieltjes transform of the limiting measure can be obtained by solving a functional equation. Under the slightly stronger assumption that $(x_{ij})$ has a strictly positive definite covariance matrix, we prove a local law for the empirical measure down to the opt…
It is a classical result of Wigner that for an hermitian matrix with independent entries on and above the diagonal, the mean empirical eigenvalue distribution converges weakly to the semicircle law as matrix size tends to infinity. In this paper, we prove analogs of Wigner's theorem for random matrices taken from all infinitesimal versions of classical symmetric spaces. This is a class of models which contains those studied by Wigner and Dyson, along with seven others arising in condensed matter physics. Like Wigner's, our results are universal in that they only depend on certain assumptions a…
