We study the statistical decision process of detecting the low-rank signal from various signal-plus-noise type data matrices, known as the spiked random matrix models. We first show that the principal component analysis can be improved by entrywise pre-transforming the data matrix if the noise is non-Gaussian, generalizing the known results for the spiked random matrix models with rank-1 signals. As an intermediate step, we find out sharp phase transition thresholds for the extreme eigenvalues of spiked random matrices, which generalize the Baik-Ben Arous-Péché (BBP) transition. We also prove the central limit theorem for the linear spectral statistics for the spiked random matrices and propose a hypothesis test based on it, which does not depend on the distribution of the signal or the noise. When the noise is non-Gaussian noise, the test can be improved with an entrywise transformation to the data matrix with additive noise. We also introduce an algorithm that estimates the rank of the signal when it is not known a priori.
We consider the problem of detecting signals in the rank-one signal-plus-noise data matrix models that generalize the spiked Wishart matrices. We show that the principal component analysis can be improved by pre-transforming the matrix entries if the noise is non-Gaussian. As an intermediate step, we prove a sharp phase transition of the largest eigenvalues of spiked rectangular matrices, which extends the Baik-Ben Arous-Péché (BBP) transition. We also propose a hypothesis test to detect the presence of signal with low computational complexity, based on the linear spectral statistics, which mi…
We consider the weak detection problem in a rank-one spiked Wigner data matrix where the signal-to-noise ratio is small so that reliable detection is impossible. We propose a hypothesis test on the presence of the signal by utilizing the linear spectral statistics of the data matrix. The test is data-driven and does not require prior knowledge about the distribution of the signal or the noise. When the noise is Gaussian, the proposed test is optimal in the sense that its error matches that of the likelihood ratio test, which minimizes the sum of the Type-I and Type-II errors. If the density of…
The present manuscript studies signal detection by likelihood ratio tests in a number of spiked random matrix models, including but not limited to Gaussian mixtures and spiked Wishart covariance matrices. We work directly with multi-spiked cases in these models and with flexible priors on the signal component that allow dependence across spikes. We derive asymptotic normality for the log-likelihood ratios when the signal-to- noise ratios are below certain thresholds. In addition, we show that the variances of the log-likelihood ratios can be asymptotically decomposed as the sums of those of a …
We consider a more generalized spiked covariance matrix $Σ$, which is a general non-definite matrix with the spiked eigenvalues scattered into a few bulks and the largest ones allowed to tend to infinity. By relaxing the matching of the 4th moment to a tail probability decay, a {\it Generalized Four Moment Theorem} (G4MT) is proposed to show the universality of the asymptotic law for the local spectral statistics of generalized spiked covariance matrices, which implies the limiting distribution of the spiked eigenvalues of the generalized spiked covariance matrix is independent of the actual d…
We study the statistical decision process of detecting the signal from a `signal+noise' type matrix model with an additive Wigner noise. We propose a hypothesis test based on the linear spectral statistics of the data matrix, which does not depend on the distribution of the signal or the noise. The test is optimal under the Gaussian noise if the signal-to-noise ratio is small, as it minimizes the sum of the Type-I and Type-II errors. Under the non-Gaussian noise, the test can be improved with an entrywise transformation to the data matrix. We also introduce an algorithm that estimates the rank…
This paper derives central limit theorems (CLTs) for general linear spectral statistics (LSS) of three important multi-spiked Hermitian random matrix ensembles. The first is the most common spiked scenario, proposed by Johnstone, which is a central Wishart ensemble with fixed-rank perturbation of the identity matrix, the second is a non-central Wishart ensemble with fixed-rank noncentrality parameter, and the third is a similarly defined non-central $F$ ensemble. These CLT results generalize our recent work to account for multiple spikes, which is the most common scenario met in practice. The …
Using the Coulomb Fluid method, this paper derives central limit theorems (CLTs) for linear spectral statistics of three "spiked" Hermitian random matrix ensembles. These include Johnstone's spiked model (i.e., central Wishart with spiked correlation), non-central Wishart with rank-one non-centrality, and a related class of non-central $F$ matrices. For a generic linear statistic, we derive simple and explicit CLT expressions as the matrix dimensions grow large. For all three ensembles under consideration, we find that the primary effect of the spike is to introduce an $O(1)$ correction term t…
The universality for the local spiked eigenvalues is a powerful tool to deal with the problems of the asymptotic law for the bulks of spiked eigenvalues of high-dimensional generalized Fisher matrices. In this paper, we focus on a more generalized spiked Fisher matrix, where $Σ_1Σ_2^{-1}$ is free of the restriction of diagonal independence, and both of the spiked eigenvalues and the population 4th moments are not necessary required to be bounded. By reducing the matching four moments constraint to a tail probability, we propose a Generalized Four Moment Theorem (G4MT) for the bulks of spiked e…
We study the asymptotic distributions of the spiked eigenvalues and the largest nonspiked eigenvalue of the sample covariance matrix under a general covariance matrix model with divergent spiked eigenvalues, while the other eigenvalues are bounded but otherwise arbitrary. The limiting normal distribution for the spiked sample eigenvalues is established. It has distinct features that the asymptotic mean relies on not only the population spikes but also the nonspikes and that the asymptotic variance in general depends on the population eigenvectors. In addition, the limiting Tracy-Widom law for …
Institut de Physique Théorique CNRS & CEA & Université Paris-Saclay（法国国家科学研究中心、原子能和替代能源委员会与巴黎-萨克雷大学理论物理研究所）；Laboratoire de Physique Statistique CNRS & Sorbonnes Universités & École Normale Supérieure, PSL University（法国国家科学研究中心、索邦大学与巴黎高等师范学院、PSL大学统计物理实验室）
Using a low-dimensional parametrization of signals is a generic and powerful way to enhance performance in signal processing and statistical inference. A very popular and widely explored type of dimensionality reduction is sparsity; another type is generative modelling of signal distributions. Generative models based on neural networks, such as GANs or variational auto-encoders, are particularly performant and are gaining on applicability. In this paper we study spiked matrix models, where a low-rank matrix is observed through a noisy channel. This problem with sparse structure of the spikes h…
We study the problem of detecting the presence of a single unknown spike in a rectangular data matrix, in a high-dimensional regime where the spike has fixed strength and the aspect ratio of the matrix converges to a finite limit. This setup includes Johnstone's spiked covariance model. We analyze the likelihood ratio of the spiked model against an "all noise" null model of reference, and show it has asymptotically Gaussian fluctuations in a region below---but in general not up to---the so-called BBP threshold from random matrix theory. Our result parallels earlier findings of Onatski et al.\ …
Many problems in statistics and machine learning require the reconstruction of a rank-one signal matrix from noisy data. Enforcing additional prior information on the rank-one component is often key to guaranteeing good recovery performance. One such prior on the low-rank component is sparsity, giving rise to the sparse principal component analysis problem. Unfortunately, there is strong evidence that this problem suffers from a computational-to-statistical gap, which may be fundamental. In this work, we study an alternative prior where the low-rank component is in the range of a trained gener…
We consider two types of spiked multivariate F distributions: a scaled distribution with the scale matrix equal to a rank-one perturbation of the identity, and a distribution with trivial scale, but rank-one non-centrality. The norm of the rank-one matrix (spike) parameterizes the joint distribution of the eigenvalues of the corresponding F matrix. We show that, for a spike located above a phase transition threshold, the asymptotic behavior of the log ratio of the joint density of the eigenvalues of the F matrix to their joint density under a local deviation from this value depends only on the…
We consider general high-dimensional spiked sample covariance models and show that their leading sample spiked eigenvalues and their linear spectral statistics are asymptotically independent when the sample size and dimension are proportional to each other. As a byproduct, we also establish the central limit theorem of the leading sample spiked eigenvalues by removing the block diagonal assumption on the population covariance matrix, which is commonly needed in the literature. Moreover, we propose consistent estimators of the $L_4$ norm of the spiked population eigenvectors. Based on these res…
The Abdus Salam International Center for Theoretical Physics（阿卜杜斯·萨拉姆国际理论物理中心）；Duke University（杜克大学）
We consider a generalization of an important class of high-dimensional inference problems, namely spiked symmetric matrix models, often used as probabilistic models for principal component analysis. Such paradigmatic models have recently attracted a lot of attention from a number of communities due to their phenomenological richness with statistical-to-computational gaps, while remaining tractable. We rigorously establish the information-theoretic limits through the proof of single-letter formulas for the mutual information and minimum mean-square error. On a technical side we improve the rece…
Consider two $p$-variate populations, not necessarily Gaussian, with covariance matrices $Σ_1$ and $Σ_2$, respectively, and let $S_1$ and $S_2$ be the sample covariances matrices from samples of the populations with degrees of freedom $T$ and $n$, respectively. When the difference $Δ$ between $Σ_1$ and $Σ_2$ is of small rank compared to $p,T$ and $n$, the Fisher matrix $F=S_2^{-1}S_1$ is called a {\em spiked Fisher matrix}. When $p,T$ and $n$ grow to infinity proportionally, we establish a phase transition for the extreme eigenvalues of $F$: when the eigenvalues of $Δ$ ({\em spikes}) are above…
This paper aims to derive asymptotical distributions of the spiked eigenvalues of the large-dimensional spiked Fisher matrices without Gaussian assumption and the restrictive assumptions on covariance matrices. We first establish invariance principle for the spiked eigenvalues of the Fisher matrix. That is, we show that the limiting distributions of the spiked eigenvalues are invariant over a large class of population distributions satisfying certain conditions. Using the invariance principle, we further established a central limit theorem (CLT) for the spiked eigenvalues. As some interesting …
This paper aims to test the number of spikes in a generalized spiked covariance matrix, the spiked eigenvalues of which may be extremely larger or smaller than the non-spiked ones. For a high-dimensional problem, we first propose a general test statistic and derive its central limit theorem by random matrix theory without a Gaussian population constraint. We then apply the result to estimate the noise variance and test the equality of the smallest roots in generalized spiked models. Simulation studies showed that the proposed test method was correctly sized, and the power outcomes showed the r…
The spiked covariance model has gained increasing popularity in high-dimensional data analysis. A fundamental problem is determination of the number of spiked eigenvalues, $K$. For estimation of $K$, most attention has focused on the use of $top$ eigenvalues of sample covariance matrix, and there is little investigation into proper ways of utilizing $bulk$ eigenvalues to estimate $K$. We propose a principled approach to incorporating bulk eigenvalues in the estimation of $K$. Our method imposes a working model on the residual covariance matrix, which is assumed to be a diagonal matrix whose en…
In this paper, we establish some new central limit theorems for certain spectral statistics of a high-dimensional sample covariance matrix under a divergent spectral norm population model. This model covers the divergent spiked population model as a special case. Meanwhile, the number of the spiked eigenvalues can either be fixed or grow to infinity. It is seen from our theorems that the divergence of population spectral norm affects the fluctuations of the linear spectral statistics in a fickle way, depending on the divergence rate.
the Wharton School（沃顿商学院）；University of Pennsylvania（宾夕法尼亚大学）
Approximate message passing (AMP) emerges as an effective iterative paradigm for solving high-dimensional statistical problems. However, prior AMP theory -- which focused mostly on high-dimensional asymptotics -- fell short of predicting the AMP dynamics when the number of iterations surpasses $o\big(\frac{\log n}{\log\log n}\big)$ (with $n$ the problem dimension). To address this inadequacy, this paper develops a non-asymptotic framework for understanding AMP in spiked matrix estimation. Built upon new decomposition of AMP updates and controllable residual terms, we lay out an analysis recipe…
Sample correlation matrices are employed ubiquitously in statistics. However, quite surprisingly, little is known about their asymptotic spectral properties for high-dimensional data, particularly beyond the case of "null models" for which the data is assumed independent. Here, considering the popular class of spiked models, we apply random matrix theory to derive asymptotic first-order and distributional results for both the leading eigenvalues and eigenvectors of sample correlation matrices. These results are obtained under high-dimensional settings for which the number of samples n and vari…
Characterizing the asymptotic distributions of eigenvectors for large random matrices poses important challenges yet can provide useful insights into a range of statistical applications. To this end, in this paper we introduce a general framework of asymptotic theory of eigenvectors (ATE) for large spiked random matrices with diverging spikes and heterogeneous variances, and establish the asymptotic properties of the spiked eigenvectors and eigenvalues for the scenario of the generalized Wigner matrix noise. Under some mild regularity conditions, we provide the asymptotic expansions for the sp…
This paper deals with the local asymptotic structure, in the sense of Le Cam's asymptotic theory of statistical experiments, of the signal detection problem in high dimension. More precisely, we consider the problem of testing the null hypothesis of sphericity of a high-dimensional covariance matrix against an alternative of (unspecified) multiple symmetry-breaking directions (\textit{multispiked} alternatives). Simple analytical expressions for the asymptotic power envelope and the asymptotic powers of previously proposed tests are derived. These asymptotic powers are shown to lie very substa…
In this paper, we establish the central limit theorem (CLT) for linear spectral statistics (LSS) of large-dimensional sample covariance matrix when the population covariance matrices are not uniformly bounded, which is a nontrivial extension of the Bai-Silverstein theorem (BST) (2004). The latter has strongly stimulated the development of high-dimensional statistics, especially the application of random matrix theory to statistics. However, the assumption of uniform boundedness of the population covariance matrices is found strongly limited to the applications of BST. The aim of this paper is …
National University of Singapore（新加坡国立大学）；Academy of Mathematics and Systems Science, Chinese Academy of Sciences（中国科学院数学与系统科学研究院）；Tencent（腾讯）
In this paper, we study the problem of principal component analysis with generative modeling assumptions, adopting a general model for the observed matrix that encompasses notable special cases, including spiked matrix recovery and phase retrieval. The key assumption is that the underlying signal lies near the range of an $L$-Lipschitz continuous generative model with bounded $k$-dimensional inputs. We propose a quadratic estimator, and show that it enjoys a statistical rate of order $\sqrt{\frac{k\log L}{m}}$, where $m$ is the number of samples. We also provide a near-matching algorithm-indep…
In this paper, we study the asymptotic behavior of the extreme eigenvalues and eigenvectors of the high dimensional spiked sample covariance matrices, in the supercritical case when a reliable detection of spikes is possible. Especially, we derive the joint distribution of the extreme eigenvalues and the generalized components of the associated eigenvectors, i.e., the projections of the eigenvectors onto arbitrary given direction, assuming that the dimension and sample size are comparably large. In general, the joint distribution is given in terms of linear combinations of finitely many Gaussi…
We study the problem of detecting an abrupt change to the signal covariance matrix. In particular, the covariance changes from a "white" identity matrix to an unknown spiked or low-rank matrix. Two sequential change-point detection procedures are presented, based on the largest and the smallest eigenvalues of the sample covariance matrix. To control false-alarm-rate, we present an accurate theoretical approximation to the average-run-length (ARL) and expected detection delay (EDD) of the detection, leveraging the extreme eigenvalue distributions from random matrix theory and by capturing a non…
In this paper, we investigate the asymptotic behavior of spiked eigenvalues of the noncentral Fisher matrix defined by ${\mathbf F}_p={\mathbf C}_n(\mathbf S_N)^{-1}$, where ${\mathbf C}_n$ is a noncentral sample covariance matrix defined by $(\mathbf Ξ+\mathbf X)(\mathbf Ξ+\mathbf X)^*/n$ and $\mathbf S_N={\mathbf Y}{\mathbf Y}^*/N$. The matrices $\mathbf X$ and $\mathbf Y$ are two independent {Gaussian} arrays, with respective $p\times n$ and $p\times N$ and the Gaussian entries of them are \textit {independent and identically distributed} (i.i.d.) with mean $0$ and variance $1$. When $p$, $…
A generalized spiked Fisher matrix is considered in this paper. We establish a criterion for the description of the support of the limiting spectral distribution of high-dimensional generalized Fisher matrix and study the almost sure limits of the sample spiked eigenvalues where the population covariance matrices are arbitrary which successively removed an unrealistic condition posed in the previous works, that is, the covariance matrices are assumed to be diagonal or diagonal block-wise structure. In addition, we also give a consistent estimator of the population spiked eigenvalues. A series …
This paper proposes a novel methodology for the online detection of changepoints in the factor structure of large matrix time series. Our approach is based on the well-known fact that, in the presence of a changepoint, a factor model can be rewritten as a model with a larger number of common factors. In turn, this entails that, in the presence of a changepoint, the number of spiked eigenvalues in the second moment matrix of the data increases. Based on this, we propose two families of procedures - one based on the fluctuations of partial sums, and one based on extreme value theory - to monitor…
The problem of estimating a spiked covariance matrix in high dimensions under Frobenius loss, and the parallel problem of estimating the noise in spiked PCA is investigated. We propose an estimator of the noise parameter by minimizing an unbiased estimator of the invariant Frobenius risk using calculus of variations. The resulting estimator is shown, using random matrix theory, to be strongly consistent and essentially asymptotically normal and minimax for the noise estimation problem. We apply the construction to construct a robust spiked covariance matrix estimator with consistent eigenvalue…
Consider the $p\times p$ matrix that is the product of a population covariance matrix and the inverse of another population covariance matrix. Suppose that their difference has a divergent rank with respect to $p$, when two samples of sizes $n$ and $T$ from the two populations are available, we construct its corresponding sample version. In the regime of high dimension where both $n$ and $T$ are proportional to $p$, we investigate the limiting laws for extreme (spiked) eigenvalues of the sample (spiked) Fisher matrix when the number of spikes is divergent and these spikes are unbounded.
We consider the online monitoring of multivariate streaming data for changes that are characterized by an unknown subspace structure manifested in the covariance matrix. In particular, we consider the covariance structure changes from an identity matrix to an unknown spiked covariance model. We assume the post-change distribution is unknown, and propose two detection procedures: the Largest-Eigenvalue Shewhart chart and the Subspace-CUSUM detection procedure. We present theoretical approximations to the average run length (ARL) and the expected detection delay (EDD) for the Largest-Eigenvalue …
We study here the so-called spiked Wigner and Wishart models, where one observes a low-rank matrix perturbed by some Gaussian noise. These models encompass many classical statistical tasks such as sparse PCA, submatrix localization, community detection or Gaussian mixture clustering. The goal of these notes is to present in a unified manner recent results (as well as new developments) on the information-theoretic limits of these spiked matrix models. We compute the minimal mean squared error for the estimation of the low-rank signal and compare it to the performance of spectral estimators and …
