We consider an Information-Plus-Noise type matrix where the Information matrix is a spiked matrix. When some eigenvalues of the random matrix separate from the bulk, we study how the corresponding eigenvectors project onto those of the spikes. Note that, in an Appendix, we present alternative versions of the earlier results of Bai and Silverstein about the lack of eigenvalues outside the support of the deterministic equivalent measure and of Capitaine about the exact separation phenomenon, where we remove some technical assumptions.
When some eigenvalues of a spiked multiplicative resp. additive deformation model of a Hermitian Wigner matrix resp. a sample covariance matrix separate from the bulk, we study how the corresponding eigenvectors project onto those of the perturbation. We point out that the inverse of the subordination function relative to the free additive resp. multiplicative convolution plays an important part in the asymptotic behavior.
We study the fluctuations associated to the a.s. convergence, established by Belinschi-Bercovici-Capitaine, of the outliers of an Hermitian polynomial in a complex Wigner matrix and a spiked deterministic real diagonal matrix. Thus, we extend the non universality phenomenon previously established for additive deformations of complex Wigner matrices, to any Hermitian polynomial. The result is described using the operator-valued subordination functions of free probability theory.
We study two spiked models of random matrices under general frameworks corresponding respectively to additive deformation of random symmetric matrices and multiplicative perturbation of random covariance matrices. In both cases, the limiting spectral measure in the direction of an eigenvector of the perturbation leads to old and new results on the coordinates of eigenvectors.
In this paper, we consider a data matrix $X_N\in\mathbb{R}^{N\times p}$ where all the rows are i.i.d. samples in $\mathbb{R}^p$ of mean zero and covariance matrix $Σ\in\mathbb{R}^{p\times p}$. Here the population matrix $Σ$ is of finite rank perturbation of the identity matrix. This is the "spiked population model" first proposed by Johnstone. As $N, p\to\infty$ but $N/p \to γ\in(1, \infty)$, for the sample covariance matrix $S_N := X_NX_N^T/N$, we establish the joint distribution of the largest and the smallest few packs of eigenvalues. Inside each pack, they will behave the same as the eigen…
In this paper, we study the eigenvalues and eigenvectors of the spiked invariant multiplicative models when the randomness is from Haar matrices. We establish the limits of the outlier eigenvalues $\widehatλ_i$ and the generalized components ($\langle \mathbf{v}, \widehat{\mathbf{u}}_i \rangle$ for any deterministic vector $\mathbb{v}$) of the outlier eigenvectors $\widehat{\mathbf{u}}_i$ with optimal convergence rates. Moreover, we prove that the non-outlier eigenvalues stick with those of the unspiked matrices and the non-outlier eigenvectors are delocalized. The results also hold near the s…
In this paper, we study the convergent limits and rates of the eigenvalues and eigenvectors for spiked sample covariance matrices whose spectrum can have multiple bulk components. Our model is an extension of Johnstone's spiked covariance matrix model. Based on our results, we can extend many statistical applications based on Johnstone's spiked covariance matrix model.
Local Law and Outlier Eigenvalues of Spiked Separable Covariance Matrices
Qiuzhen College, Tsinghua University（清华大学邱氏书院）；Department of Probability and Statistics, School of Mathematical Sciences, Peking University（北京大学数学科学学院概率统计系）
We prove large deviations principles for spectral measures of perturbed (or spiked) matrix models in the direction of an eigenvector of the perturbation. In each model under study, we provide two approaches, one of which relying on large deviations principle of unperturbed models derived in the previous work "Sum rules via large deviations" (Gamboa-Nagel-Rouault, JFA, 2016).
A class of robust estimators of scatter applied to information-plus-impulsive noise samples is studied, where the sample information matrix is assumed of low rank; this generalizes the study of (Couillet et al., 2013b) to spiked random matrix models. It is precisely shown that, as opposed to sample covariance matrices which may have asymptotically unbounded (eigen-)spectrum due to the sample impulsiveness, the robust estimator of scatter has bounded spectrum and may contain isolated eigenvalues which we fully characterize. We show that, if found beyond a certain detectability threshold, these …
In this paper, we study the asymptotic behavior of the extreme eigenvalues and eigenvectors of the spiked covariance matrices, in the supercritical regime. Specifically, we derive the joint distribution of the extreme eigenvalues and the generalized components of their associated eigenvectors in this regime.
We introduce a class of separable sample covariance matrices of the form $\widetilde{\mathcal{Q}}_1:=\widetilde A^{1/2} X \widetilde B X^* \widetilde A^{1/2}.$ Here $\widetilde{A}$ and $\widetilde{B}$ are positive definite matrices whose spectrums consist of bulk spectrums plus several spikes, i.e. larger eigenvalues that are separated from the bulks. Conceptually, we call $\widetilde{\mathcal{Q}}_1$ a \emph{spiked separable covariance matrix model}. On the one hand, this model includes the spiked covariance matrix as a special case with $\widetilde{B}=I$. On the other hand, it allows for more…
We consider matrices formed by a random $N\times N$ matrix drawn from the Gaussian Orthogonal Ensemble (or Gaussian Unitary Ensemble) plus a rank-one perturbation of strength $θ$, and focus on the largest eigenvalue, $x$, and the component, $u$, of the corresponding eigenvector in the direction associated to the rank-one perturbation. We obtain the large deviation principle governing the atypical joint fluctuations of $x$ and $u$. Interestingly, for $θ>1$, in large deviations characterized by a small value of $u$, i.e. $u<1-1/θ$, the second-largest eigenvalue pops out from the Wigner semi-circ…
In this paper, we investigate the fluctuations of a unit eigenvector associated to an outlier in the spectrum of a spiked $N\times N$ complex Deformed Wigner matrix $M_N$: $M_N =W_N/\sqrt{N} + A_N$ where $W_N$ is an $N \times N$ Hermitian Wigner matrix whose entries have a law $μ$ satisfying a Poincaré inequality and the matrix $A_N$ is a block diagonal matrix, with an eigenvalue $θ$ of multiplicity one, generating an outlier in the spectrum of $M_N$. We prove that the fluctuations of the norm of the projection of a unit eigenvector corresponding to the outlier of $M_N$ onto a unit eigenvector…
In this paper, we derive a joint central limit theorem for random vector whose components are function of random sesquilinear forms. This result is a natural extension of the existing central limit theory on random quadratic forms. We also provide applications in random matrix theory related to large-dimensional spiked population models. For the first application, we find the joint distribution of grouped extreme sample eigenvalues correspond to the spikes. And for the second application, under the assumption that the population covariance matrix is diagonal with $k$ (fixed) simple spikes, we …
We study principal components analyses in multivariate random and mixed effects linear models, assuming a spherical-plus-spikes structure for the covariance matrix of each random effect. We characterize the behavior of outlier sample eigenvalues and eigenvectors of MANOVA variance components estimators in such models under a high-dimensional asymptotic regime. Our results show that an aliasing phenomenon may occur in high dimensions, in which eigenvalues and eigenvectors of the MANOVA estimate for one variance component may be influenced by the other components. We propose an alternative proce…
Let $\mathbf{W}\in\mathbb{C}^{n\times n}$ be a {\it single-spiked} Wishart matrix in the class $\mathbf{W}\sim \mathcal{CW}_n(m,\mathbf{I}_n+ θ\mathbf{v}\mathbf{v}^\dagger) $ with $m\geq n$, where $\mathbf{I}_n$ is the $n\times n$ identity matrix, $\mathbf{v}\in\mathbb{C}^{n\times 1}$ is an arbitrary vector with unit Euclidean norm, $θ\geq 0$ is a non-random parameter, and $(\cdot)^\dagger$ represents the conjugate-transpose operator. Let $\mathbf{u}_1$ and $\mathbf{u}_n$ denote the eigenvectors corresponding to the samllest and the largest eigenvalues of $\mathbf{W}$, respectively. This paper…
This text is about spiked models of non Hermitian random matrices. More specifically, we consider matrices of the type $A+P$, where the rank of $P$ stays bounded as the dimension goes to infinity and where the matrix $A$ is a non Hermitian random matrix, satisfying an isotropy hypothesis: its distribution is invariant under the left and right actions of the unitary group. The macroscopic eigenvalue distribution of such matrices is governed by the so called Single Ring Theorem, due to Guionnet, Krishnapur and Zeitouni. We first prove that if $P$ has some eigenvalues out of the maximal circle of…
We consider the eigenvalues and eigenvectors of small rank perturbations of random $N\times N$ matrices. We allow the rank of perturbation $M$ increases with $N$, and the only assumption is $M=o(N)$. In both additive and multiplicative perturbation models, we prove rigidity results for the outliers of the perturbed random matrices. Based on the rigidity results we derive the empirical distribution of outliers of the perturbed random matrices. We also compute the appropriate projection of eigenvectors corresponding to the outliers of the perturbed random matrices, which are approximate eigenvec…
For any family of $N\times N$ random matrices $(\mathbf{A}_k)_{k\in K}$ which is invariant, in law, under unitary conjugation, we give general sufficient conditions for central limit theorems for random variables of the type $\operatorname{Tr}(\mathbf{A}_k \mathbf{M})$, where the matrix $\mathbf{M}$ is deterministic (such random variables include for example the normalized matrix entries of the $\mathbf{A}_k$'s). A consequence is the asymptotic independence of the projection of the matrices $\mathbf{A}_k$ onto the subspace of null trace matrices from their projections onto the orthogonal of th…
We consider large Information-Plus-Noise type matrices of the form $M_N=(σ\frac{X_N}{\sqrt{N}}+A_N)(σ\frac{X_N}{\sqrt{N}}+A_N)^*$ where $X_N$ is an $n \times N$ ($n\leq N)$ matrix consisting of independent standardized complex entries, $A_N$ is an $n \times N$ nonrandom matrix and $σ>0$. As $N$ tends to infinity, if $n/N \rightarrow c\in ]0,1]$ and if the empirical spectral measure of $A_N A_N^*$ converges weakly to some compactly supported probability distribution $ν\neq δ_0$, Dozier and Silverstein established that almost surely the empirical spectral measure of $M_N$ converges weakly toward…
It is known that in various random matrix models, large perturbations create outlier eigenvalues which lie, asymptotically, in the complement of the support of the limiting spectral density. This paper is concerned with fluctuations of these outlier eigenvalues of iid matrices $X_n$ under bounded rank and bounded operator norm perturbations $A_n$, namely with $λ(\frac{X_n}{\sqrt{n}}+A_n)-λ(A_n)$. The perturbations we consider are allowed to be of arbitrary Jordan type and have (left and right) eigenvectors satisfying a mild condition. We obtain the joint convergence of the (normalized) asympto…
In this paper, we study the random matrix model of Gaussian Unitary Ensemble (GUE) with fixed-rank (aka spiked) external source. We will focus on the critical regime of the Baik-Ben Arous-Péché (BBP) phase transition and establish the distribution of the eigenvectors associated with the leading eigenvalues. The distribution is given in terms of a determinantal point process with extended Airy kernel. Our result can be regarded as an eigenvector counterpart of the BBP eigenvalue phase transition (arXiv:math/0403022). The derivation of the distribution makes use of the recently re-discovered eig…
We consider the eigenvalues and eigenvectors of finite, low rank perturbations of random matrices. Specifically, we prove almost sure convergence of the extreme eigenvalues and appropriate projections of the corresponding eigenvectors of the perturbed matrix for additive and multiplicative perturbation models. The limiting non-random value is shown to depend explicitly on the limiting eigenvalue distribution of the unperturbed random matrix and the assumed perturbation model via integral transforms that correspond to very well known objects in free probability theory that linearize non-commuta…
We introduce a class of $M \times M$ sample covariance matrices $\mathcal Q$ which subsumes and generalizes several previous models. The associated population covariance matrix $Σ= \mathbb E \cal Q$ is assumed to differ from the identity by a matrix of bounded rank. All quantities except the rank of $Σ- I_M$ may depend on $M$ in an arbitrary fashion. We investigate the principal components, i.e.\ the top eigenvalues and eigenvectors, of $\mathcal Q$. We derive precise large deviation estimates on the generalized components $\langle \mathbf w, \boldsymbol ξ_i \rangle$ of the outlier and non-out…
Recovering low-rank structures via eigenvector perturbation analysis is a common problem in statistical machine learning, such as in factor analysis, community detection, ranking, matrix completion, among others. While a large variety of bounds are available for average errors between empirical and population statistics of eigenvectors, few results are tight for entrywise analyses, which are critical for a number of problems such as community detection. This paper investigates entrywise behaviors of eigenvectors for a large class of random matrices whose expectations are low-rank, which helps …
We study the principal components of covariance estimators in multivariate mixed-effects linear models. We show that, in high dimensions, the principal eigenvalues and eigenvectors may exhibit bias and aliasing effects that are not present in low-dimensional settings. We derive the first-order limits of the principal eigenvalue locations and eigenvector projections in a high-dimensional asymptotic framework, allowing for general population spectral distributions for the random effects and extending previous results from a more restrictive spiked model. Our analysis uses free probability techni…
We derive the joint asymptotic distribution of the outlier eigenvalues of an additively deformed Wigner matrix $H$. Our only assumptions on the deformation are that its rank be fixed and its norm bounded. Our results extend those of [The isotropic semicircle law and deformation of Wigner matrices. Preprint] by admitting overlapping outliers and by computing the joint distribution of all outliers. In particular, we give a complete description of the failure of universality first observed in [Ann. Probab. 37 (2009) 1-47; Ann. Inst. Henri Poincaré Probab. Stat. 48 (1013) 107-133; Free convolution…
We study the distribution of the outliers in the spectrum of finite rank deformations of Wigner random matrices. We assume that the matrix entries have finite fourth moment and extend the results by Capitaine, Donati-Martin, and Féral for perturbations whose eigenvectors are delocalized.
A central problem of random matrix theory is to understand the eigenvalues of spiked random matrix models, introduced by Johnstone, in which a prominent eigenvector (or "spike") is planted into a random matrix. These distributions form natural statistical models for principal component analysis (PCA) problems throughout the sciences. Baik, Ben Arous and Peche showed that the spiked Wishart ensemble exhibits a sharp phase transition asymptotically: when the spike strength is above a critical threshold, it is possible to detect the presence of a spike based on the top eigenvalue, and below the t…
We consider the eigenvalues and eigenvectors of matrices of the form M + P, where M is an n by n Wigner random matrix and P is an arbitrary n by n deterministic matrix with low rank. In general, we show that none of the eigenvalues of M + P need be real, even when P has rank one. We also show that, except for a few outlier eigenvalues, most of the eigenvalues of M + P are within 1/n of the real line, up to small order corrections. We also prove a new result quantifying the outlier eigenvalues for multiplicative perturbations of the form S ( I + P ), where S is a sample covariance matrix and I …
A central problem of random matrix theory is to understand the eigenvalues of spiked random matrix models, in which a prominent eigenvector is planted into a random matrix. These distributions form natural statistical models for principal component analysis (PCA) problems throughout the sciences. Baik, Ben Arous and Péché showed that the spiked Wishart ensemble exhibits a sharp phase transition asymptotically: when the signal strength is above a critical threshold, it is possible to detect the presence of a spike based on the top eigenvalue, and below the threshold the top eigenvalue provides …
We revisit the problem of an elastic line (e.g. a vortex line in a superconductor) subject to both columnar disorder and point disorder in dimension $d=1+1$. Upon applying a transverse field, a delocalization transition is expected, beyond which the line is tilted macroscopically. We investigate this transition in the fixed tilt angle ensemble and within a one-way model where backward jumps are neglected. From recent results about directed polymers and their connections to random matrix theory, we find that for a single line and a single strong defect this transition in presence of point disor…
Consider a real diagonal deterministic matrix $X_n$ of size $n$ with spectral measure converging to a compactly supported probability measure. We perturb this matrix by adding a random finite rank matrix, with delocalized eigenvectors. We show that the joint law of the extreme eigenvalues of the perturbed model satisfies a large deviation principle in the scale $n$, with a good rate function given by a variational formula. We tackle both cases when the extreme eigenvalues of $X_n$ converge to the edges of the support of the limiting measure and when we allow some eigenvalues of $X_n$, that we …
For a generalization of Johnstone's spiked model, a covariance matrix with eigenvalues all one but $M$ of them, the number of features $N$ comparable to the number of samples $n: N=N(n), M=M(n), γ^{-1} \leq \frac{N}{n} \leq γ$ where $γ\in (0,\infty),$ we obtain consistency rates in the form of CLTs for separated spikes tending to infinity fast enough whenever $M$ grows slightly slower than $n: \lim_{n \to \infty}{\frac{\sqrt{\log{n}}}{\log{\frac{n}{M(n)}}}}=0.$ Our results fill a gap in the existing literature in which the largest range covered for the number of spikes has been $o(n^{1/6})$ an…
We analyse the spectrum of additive finite-rank deformations of $N \times N$ Wigner matrices $H$. The spectrum of the deformed matrix undergoes a transition, associated with the creation or annihilation of an outlier, when an eigenvalue $d_i$ of the deformation crosses a critical value $\pm 1$. This transition happens on the scale $|d_i| - 1 \sim N^{-1/3}$. We allow the eigenvalues $d_i$ of the deformation to depend on $N$ under the condition $|\abs{d_i} - 1| \geq (\log N)^{C \log \log N} N^{-1/3}$. We make no assumptions on the eigenvectors of the deformation. In the limit $N \to \infty$, we …
The asymptotic freeness of independent unitarily invariant $N\times N$ random matrices holds in expectation up to $O(N^{-2})$. An already known consequence is the infinitesimal freeness in expectation. We put in evidence another consequence for unitarily invariant random matrices: the almost sure asymptotic freeness of type $B$. As byproducts, we recover the asymptotic cyclic monotonicity, and we get the asymptotic conditional freeness. In particular, the eigenvector empirical spectral distribution of the sum of two randomly rotated random matrices converges towards the conditionally free conv…
This paper is devoted to the estimation of the minimal dimension P of the state-space realizations of a high-dimensional time series y, defined as a noisy version (the noise is white and Gaussian) of a useful signal with low rank rational spectral density, in the high-dimensional asymptotic regime where the number of available samples N and the dimension of the time series M converge towards infinity at the same rate. In the classical low-dimensional regime, P is estimated as the number of significant singular values of the empirical autocovariance matrix between the past and the future of y, …
