We consider settings where the observations are drawn from a zero-mean multivariate (real or complex) normal distribution with the population covariance matrix having eigenvalues of arbitrary multiplicity. We assume that the eigenvectors of the population covariance matrix are unknown and focus on inferential procedures that are based on the sample eigenvalues alone (i.e., "eigen-inference"). Results found in the literature establish the asymptotic normality of the fluctuation in the trace of powers of the sample covariance matrix. We develop concrete algorithms for analytically computing the limiting quantities and the covariance of the fluctuations. We exploit the asymptotic normality of the trace of powers of the sample covariance matrix to develop eigenvalue-based procedures for testing and estimation. Specifically, we formulate a simple test of hypotheses for the population eigenvalues and a technique for estimating the population eigenvalues in settings where the cumulative distribution function of the (nonrandom) population eigenvalues has a staircase structure. Monte Carlo simulations are used to demonstrate the superiority of the proposed methodologies over classical techniques and the robustness of the proposed techniques in high-dimensional, (relatively) small sample size settings. The improved performance results from the fact that the proposed inference procedures are "global" (in a sense that we describe) and exploit "global" information thereby overcoming the inherent biases that cripple classical inference procedures which are "local" and rely on "local" information.
This study derives a new property of the Wishart distribution when the degree-of-freedom and the size of the matrix parameter of the distribution grow simultaneoulsy. Particularly, the asymptotic normality of the product of four independent Wishart matrices is shown under a high dimensional asymptotic regime. As an application of the result, a statistical test procedure for the common principal components hypothesis is proposed. For this problem, the proposed test statistic is asymptotically normal under the null hypothesis. In addition, the proposed test statistic diverges to positive infinit…
This paper deals with the asymptotic distribution of Wishart matrix and its application to the estimation of the population matrix parameter when the population eigenvalues are block-wise infinitely dispersed. We show that the appropriately normalized eigenvectors and eigenvalues asymptotically generate two Wishart matrices and one normally distributed random matrix, which are mutually independent. For a family of orthogonally equivariant estimators, we calculate the asymptotic risks with respect to the entropy or the quadratic loss function and derive the asymptotically best estimator among t…
In this paper we derive some new and practical results on testing and interval estimation problems for the population eigenvalues of a Wishart matrix based on the asymptotic theory for block-wise infinite dispersion of the population eigenvalues. This new type of asymptotic theory has been developed by the present authors in Takemura and Sheena (2005) and Sheena and Takemura (2007a,b) and in these papers it was applied to point estimation problem of population covariance matrix in a decision theoretic framework. In this paper we apply it to some testing and interval estimation problems. We sho…
The space of positive definite symmetric matrices has been studied extensively as a means of understanding dependence in multivariate data along with the accompanying problems in statistical inference. Many books and papers have been written on this subject, and more recently there has been considerable interest in high-dimensional random matrices with particular emphasis on the distribution of certain eigenvalues. With the availability of modern data acquisition capabilities, smoothing or nonparametric techniques are required that go beyond those applicable only to data arising in Euclidean s…
We study the central limit theorem for sums of independent tensor powers, $\frac{1}{\sqrt{d}}\sum\limits_{i=1}^d X_i^{\otimes p}$. We focus on the high-dimensional regime where $X_i \in \mathbb{R}^n$ and $n$ may scale with $d$. Our main result is a proposed threshold for convergence. Specifically, we show that, under some regularity assumption, if $n^{2p-1}\gg d$, then the normalized sum converges to a Gaussian. The results apply, among others, to symmetric uniform log-concave measures and to product measures. This generalizes several results found in the literature. Our main technique is a no…
We introduce a new method to qualify the goodness of fit parameter estimation of compound Wishart models. Our method based on the free deterministic equivalent Z-score, which we introduce in this paper. Furthermore, an application to two dimensional autoregressive moving-average model is provided. Our proposal method is a generalization of statistical hypothesis testing to one dimensional moving average model based on fluctuations of real compound Wishart matrices, which is a recent result by Hasegawa, Sakuma and Yoshida.
Geostatistical modeling for continuous point-referenced data has been extensively applied to neuroimaging because it produces efficient and valid statistical inference. However, diffusion tensor imaging (DTI), a neuroimaging characterizing the brain structure produces a positive definite (p.d.) matrix for each voxel. Current geostatistical modeling has not been extended to p.d. matrices because introducing spatial dependence among positive definite matrices properly is challenging. In this paper, we use the spatial Wishart process, a spatial stochastic process (random field) where each p.d. ma…
We derive concentration inequalities for the spectral measure of large random matrices, allowing for certain forms of dependence. Our main focus is on empirical covariance (Wishart) matrices, but general symmetric random matrices are also considered.
A holonomic system for the probability density function of the largest eigenvalue of a non-central complex Wishart distribution with identity covariance matrix is derived. Furthermore a new determinantal formula for the probability density function is derived (for m=2,3) or conjectured.
In this study, we derive the exact distributions of eigenvalues of a singular Wishart matrix under an elliptical model. We define generalized heterogeneous hypergeometric functions with two matrix arguments and provide convergence conditions for these functions. The joint density of eigenvalues and the distribution function of the largest eigenvalue for a singular elliptical Wishart matrix are represented by these functions. Numerical computations for the distribution of the largest eigenvalue were conducted under the matrix-variate $t$ and Kotz-type models.
We study the distribution of the ratio of two central Wishart matrices with different covariance matrices. We first derive the density function of a particular matrix form of the ratio and show that its cumulative distribution function can be expressed in terms of the hypergeometric function 2F1 of a matrix argument. Then we apply the holonomic gradient method for numerical evaluation of the hypergeometric function. This approach enables us to compute the power function of Roy's maximum root test for testing the equality of two covariance matrices.
This paper discusses certain properties of heterogeneous hypergeometric functions with two matrix arguments. These functions are newly defined but have already appeared in statistical literature and are useful when dealing with the derivation of certain distributions for the eigenvalues of singular beta-Wishart matrices. The joint density function of the eigenvalues and the distribution of the largest eigenvalue can be expressed in terms of certain heterogeneous hypergeometric functions. Exact computation of the distribution of the largest eigenvalue is conducted here for a real case.
This paper proposes a unified approach that enables the Wishart distribution to be studied simultaneously in the real, complex, quaternion and octonion cases. In particular, the noncentral generalised Wishart distribution, the joint density of the eigenvalues and the distribution of the maximum eigenvalue are obtained for real normed division algebras.
By using a symbolic method, known in the literature as the classical umbral calculus, the trace of a non-central Wishart random matrix is represented as the convolution of the trace of its central component and of a formal variable involving traces of its non-centrality matrix. Thanks to this representation, the moments of this random matrix are proved to be a Sheffer polynomial sequence, allowing us to recover several properties. The multivariate symbolic method generalizes the employment of Sheffer representation and a closed form formula for computing joint moments and cumulants (also norma…
We introduce a family of matrices with non-commutative entries that generalize the classical real Wishart matrices. With the help of the Brauer product, we derive a non-asymptotic expression for the moments of traces of monomials in such matrices; the expression is quite similar to the formula derived in our previous work for independent complex Wishart matrices. We then analyze the fluctuations about the Marchenko-Pastur law. We show that after centering by the mean, traces of real symmetric polynomials in q-Wishart matrices converge in distribution, and we identify the asymptotic law as the …
We consider two types of averaging of complex covariance matrices, a sample mean (average) and the sample Fréchet mean. We analyse the performance of these quantities as estimators for the true covariance matrix via `intrinsic' versions of bias and mean square error, a methodology which takes account of geometric structure. We derive simple expressions for the intrinsic bias in both cases, and the simple average is seen to be preferable. The same is true for the asymptotic Riemannian risk, and for the Riemannian risk itself in the scalar case. Combined with a similar preference for the simple …
Let A be a p-variate real Wishart matrix on n degrees of freedom with identity covariance. The distribution of the largest eigenvalue in A has important applications in multivariate statistics. Consider the asymptotics when p grows in proportion to n, it is known from Johnstone (2001) that after centering and scaling, these distributions approach the orthogonal Tracy-Widom law for real-valued data, which can be numerically evaluated and tabulated in software. Under the same assumption, we show that more carefully chosen centering and scaling constants improve the accuracy of the distributional…
We give an approximate formula for the distribution of the largest eigenvalue of real Wishart matrices by the expected Euler characteristic method for the general dimension. The formula is expressed in terms of a definite integral with parameters. We derive a differential equation satisfied by the integral for the $2 \times 2$ matrix case and perform a numerical analysis of it.
We calculate the `one-point function', meaning the marginal probability density function for any single eigenvalue, of real and complex Wishart correlation matrices. No explicit expression had been obtained for the real case so far. We succeed in doing so by using supersymmetry techniques to express the one-point function of real Wishart correlation matrices as a twofold integral. The result can be viewed as a resummation of a series of Jack polynomials in a non-trivial case. We illustrate our formula by numerical simulations. We also rederive a known expression for the one-point function of c…
For a random matrix following a Wishart distribution, we derive formulas for the expectation and the covariance matrix of compound matrices. The compound matrix of order $m$ is populated by all $m\times m$-minors of the Wishart matrix. Our results yield first and second moments of the minors of the sample covariance matrix for multivariate normal observations. This work is motivated by the fact that such minors arise in the expression of constraints on the covariance matrix in many classical multivariate problems.
Consider the classical Gaussian unitary ensemble of size $N$ and the real Wishart ensemble $W_N(n,I)$. In the limits as $N \to \infty$ and $N/n \to γ> 0$, the expected number of eigenvalues that exit the upper bulk edge is less than one, 0.031 and 0.170 respectively, the latter number being independent of $γ$. These statements are consequences of quantitative bounds on tail sums of eigenvalues outside the bulk which are established here for applications in high dimensional covariance matrix estimation.
We study the behavior of a real $p$-dimensional Wishart random matrix with $n$ degrees of freedom when $n,p\rightarrow\infty$ but $p/n\rightarrow 0$. We establish the existence of phase transitions when $p$ grows at the order $n^{(K+1)/(K+3)}$ for every $k\in\mathbb{N}$, and derive expressions for approximating densities between every two phase transitions. To do this, we make use of a novel tool we call the G-transform of a distribution, which is closely related to the characteristic function. We also derive an extension of the $t$-distribution to the real symmetric matrices, which naturally …
This note reports partial results related to the Gaussian product inequality (GPI) conjecture for the joint distribution of traces of Wishart matrices. In particular, several GPI-related results from Wei (2014) and Liu et al. (2015) are extended in two ways: by replacing the power functions with more general classes of functions, and by replacing the usual Gaussian and multivariate gamma distributional assumptions by the more general trace-Wishart distribution assumption. These findings suggest that a Kronecker product form of the GPI holds for diagonal blocks of any Wishart distribution.
Let $\mathbf{W}$ be a correlated complex non-central Wishart matrix defined through $\mathbf{W}=\mathbf{X}^H\mathbf{X}$, where $\mathbf{X}$ is $n\times m \, (n\geq m)$ complex Gaussian with non-zero mean $\boldsymbolΥ$ and non-trivial covariance $\boldsymbolΣ$. We derive exact expressions for the cumulative distribution functions (c.d.f.s) of the extreme eigenvalues (i.e., maximum and minimum) of $\mathbf{W}$ for some particular cases. These results are quite simple, involving rapidly converging infinite series, and apply for the practically important case where $\boldsymbolΥ$ has rank one. We…
The sum of independent Wishart matrices, taken from distributions with unequal covariance matrices, plays a crucial role in multivariate statistics, and has applications in the fields of quantitative finance and telecommunication. However, analytical results concerning the corresponding eigenvalue statistics have remained unavailable, even for the sum of two Wishart matrices. This can be attributed to the complicated and rotationally noninvariant nature of the matrix distribution that makes extracting the information about eigenvalues a nontrivial task. Using a generalization of the Harish-Cha…
We derive efficient recursive formulas giving the exact distribution of the largest eigenvalue for finite dimensional real Wishart matrices and for the Gaussian Orthogonal Ensemble (GOE). In comparing the exact distribution with the limiting distribution of large random matrices, we also found that the Tracy-Widom law can be approximated by a properly scaled and shifted Gamma distribution, with great accuracy for the values of common interest in statistical applications.
In this paper, we obtain a property of the expectation of the inverse of compound Wishart matrices which results from their orthogonal invariance. Using this property as well as results from random matrix theory (RMT), we derive the asymptotic effect of the noise induced by estimating the covariance matrix on computing the risk of the optimal portfolio. This in turn enables us to get an asymptotically unbiased estimator of the risk of the optimal portfolio not only for the case of independent observations but also in the case of correlated observations. This improvement provides a new approach…
Wishart random matrix theory is of major importance for the analysis of correlated time series. The distribution of the smallest eigenvalue for Wishart correlation matrices is particularly interesting in many applications. In the complex and in the real case, we calculate it exactly for arbitrary empirical eigenvalues, i.e., for fully correlated Gaussian Wishart ensembles. To this end, we derive certain dualities of matrix models in ordinary space. We thereby completely avoid the otherwise unsurmountable problem of computing a highly non-trivial group integral. Our results are compact and much…
Wishart correlation matrices are the standard model for the statistical analysis of time series. The ensemble averaged eigenvalue density is of considerable practical and theoretical interest. For complex time series and correlation matrices, the eigenvalue density is known exactly. In the real case, however, a fundamental mathematical obstacle made it forbidingly complicated to obtain exact results. We use the supersymmetry method to fully circumvent this problem. We present an exact formula for the eigenvalue density in the real case in terms of twofold integrals and finite sums.
Recently, D. Wang has devised a new contour integral based method to simplify certain matrix integrals. Capitalizing on that approach, we derive a new expression for the probability density function (p.d.f.) of the joint eigenvalues of a complex non-central Wishart matrix with a rank-1 mean. The resulting functional form in turn enables us to use powerful classical orthogonal polynomial techniques in solving three problems related to the non-central Wishart matrix. To be specific, for an $n\times n$ complex non-central Wishart matrix $\mathbf{W}$ with $m$ degrees of freedom ($m\geq n$) and a r…
The distributions of the largest and the smallest eigenvalues of a $p$-variate sample covariance matrix $S$ are of great importance in statistics. Focusing on the null case where $nS$ follows the standard Wishart distribution $W_p(I,n)$, we study the accuracy of their scaling limits under the setting: $n/p\rightarrow γ\in(0,\infty)$ as $n\rightarrow \infty$. The limits here are the orthogonal Tracy--Widom law and its reflection about the origin. With carefully chosen rescaling constants, the approximation to the rescaled largest eigenvalue distribution by the limit attains accuracy of order ${…
We derive the probability that all eigenvalues of a random matrix $\bf M$ lie within an arbitrary interval $[a,b]$, $ψ(a,b)\triangleq\Pr\{a\leqλ_{\min}({\bf M}), λ_{\max}({\bf M})\leq b\}$, when $\bf M$ is a real or complex finite dimensional Wishart, double Wishart, or Gaussian symmetric/Hermitian matrix. We give efficient recursive formulas allowing the exact evaluation of $ψ(a,b)$ for Wishart matrices, even with large number of variates and degrees of freedom. We also prove that the probability that all eigenvalues are within the limiting spectral support (given by the Mar{č}enko-Pastur or …
To explore the limits of a stochastic gradient method, it may be useful to consider an example consisting of an infinite number of quadratic functions. In this context, it is appropriate to determine the expected value and the covariance matrix of the stochastic noise, i.e. the difference of the true gradient and the approximated gradient generated from a finite sample. When specifying the covariance matrix, the expected value of a quadratic form QBQ is needed, where Q is a Wishart distributed random matrix and B is an arbitrary fixed symmetric matrix. After deriving an expression for E(QBQ) a…
Data sets collected at different times and different observing points can possess correlations at different times $and$ at different positions. The doubly correlated Wishart model takes both into account. We calculate the eigenvalue density of the Wishart correlation matrices using supersymmetry. In the complex case we obtain a new closed form expression which we compare to previous results in the literature. In the more relevant and much more complicated real case we derive an expression for the density in terms of a fourfold integral. Finally, we calculate the density in the limit of large c…
Let X be a n*p matrix and l_1 the largest eigenvalue of the covariance matrix X^{*}*X. The "null case" where X_{i,j} are independent Normal(0,1) is of particular interest for principal component analysis. For this model, when n, p tend to infinity and n/p tends to gamma in (0,\infty), it was shown in Johnstone (2001) that l_1, properly centered and scaled, converges to the Tracy-Widom law. We show that with the same centering and scaling, the result is true even when p/n or n/p tends to infinity. The derivation uses ideas and techniques quite similar to the ones presented in Johnstone (2001). …
We study the asymptotic behavior of eigenvalues of large complex correlated Wishart matrices at the edges of the limiting spectrum. In this setting, the support of the limiting eigenvalue distribution may have several connected components. Under mild conditions for the population matrices, we show that for every generic positive edge of that support, there exists an extremal eigenvalue which converges almost surely toward that edge and fluctuates according to the Tracy-Widom law at the scale $N^{2/3}$. Moreover, given several generic positive edges, we establish that the associated extremal ei…
The outage probability of maximal-ratio combining (MRC) for a multiple-input multiple-output (MIMO) wireless communications system under Rician fading is given by the cumulative distribution function (CDF) for the largest eigenvalue of a complex noncentral Wishart matrix. This CDF has previously been expressed as a determinant whose elements are integrals of a confluent hypergeometric function. For the determinant elements, conventional evaluation approaches, e.g., truncation of infinite series ensuing from the hypergeometric function or numerical integration, can be unreliable and slow even f…
For the correlated Gaussian Wishart ensemble we compute the distribution of the smallest eigenvalue and a related gap probability.We obtain exact results for the complex (β=2) and for the real case (β=1). For a particular set of empirical correlation matrices we find universality in the spectral density, for both real and complex ensembles and all kinds of rectangularity. We calculate the asymptotic and universal results for the gap probability and the distribution of the smallest eigenvalue. We use the Supersymmetry method, in particular the generalized Hubbard-Stratonovich transformation and…
Extreme principal minors of Wishart and deformed GOE matrices
School of Data Science, The Chinese University of Hong Kong, Shenzhen（香港中文大学深圳数据科学学院）；Department of Statistics and Data Science, University of Texas, Austin（德克萨斯大学奥斯汀分校统计与数据科学系）
We consider four nontrivial ensembles involving Gaussian Wigner and Wishart matrices. These are relevant to problems ranging from multiantenna communication to random supergravity. We derive the matrix probability density, as well as the eigenvalue densities for these ensembles. In all cases the joint eigenvalue density exhibits a biorthogonal structure. A determinantal representation, based on a generalization of Andréief's integration formula, is used to compactly express the $r$-point correlation function of eigenvalues. This representation circumvents the complications encountered in the u…
We define an indefinite Wishart matrix as a matrix of the form A=W^{T}WΣ, where Σis an indefinite diagonal matrix and W is a matrix of independent standard normals. We focus on the case where W is L by 2 which has engineering applications. We obtain the distribution of the ratio of the eigenvalues of A. This distribution can be "folded" to give the distribution of the condition number. We calculate formulas for W real (β=1), complex (β=2), quaternionic (β=4) or any ghost 0<β<\infty. We then corroborate our work by comparing them against numerical experiments.
The correlated Wishart model provides a standard tool for the analysis of correlations in a rich variety of systems. Although much is known for complex correlation matrices, the empirically much more important real case still poses substantial challenges. We put forward a new approach, which maps arbitrary statistical quantities, depending on invariants only, to invariant Hermitian matrix models. For completeness we also include the quaternion case and deal with all three cases in a unified way. As an important application, we study the statistics of the largest eigenvalue and its limiting dis…
We consider high dimensional Wishart matrices $\mathbb{X} \mathbb{X}^{\top}$ where the entries of $\mathbb{X} \in {\mathbb{R}^{n \times d}}$ are i.i.d. from a log-concave distribution. We prove an information theoretic phase transition: such matrices are close in total variation distance to the corresponding Gaussian ensemble if and only if $d$ is much larger than $n^3$. Our proof is entropy-based, making use of the chain rule for relative entropy along with the recursive structure in the definition of the Wishart ensemble. The proof crucially relies on the well known relation between Fisher i…
The statistical behaviour of the smallest eigenvalue has important implications for systems which can be modeled using a Wishart-Laguerre ensemble, the regular one or the fixed trace one. For example, the density of the smallest eigenvalue of the Wishart-Laguerre ensemble plays a crucial role in characterizing multiple channel telecommunication systems. Similarly, in the quantum entanglement problem, the smallest eigenvalue of the fixed trace ensemble carries information regarding the nature of entanglement. For real Wishart-Laguerre matrices, there exists an elegant recurrence scheme suggeste…
Vinberg cones and the ambient vector spaces are important in modern statistics of sparse models and of graphical models. The aim of this paper is to study eigenvalue distributions of Gaussian, Wigner and covariance matrices related to growing Vinberg matrices, corresponding to growing daisy graphs. For Gaussian or Wigner ensembles, we give an explicit formula for the limiting distribution. For Wishart ensembles defined naturally on Vinberg cones, their limiting Stieltjes transforms, support and atom at 0 are described explicitly in terms of the Lambert-Tsallis functions, which are defined by u…
The aim of this note is to provide a pedagogical survey of the recent works by the authors ( arXiv:1409.7548 and arXiv:1507.06013) concerning the local behavior of the eigenvalues of large complex correlated Wishart matrices at the edges and cusp points of the spectrum: Under quite general conditions, the eigenvalues fluctuations at a soft edge of the limiting spectrum, at the hard edge when it is present, or at a cusp point, are respectively described by mean of the Airy kernel, the Bessel kernel, or the Pearcey kernel. Moreover, the eigenvalues fluctuations at several soft edges are asymptot…
A recent line of work has studied the relationship between the Wishart matrix $X^\top X$, where $X\in \mathbb{R}^{d\times n}$ has i.i.d. standard Gaussian entries, and the corresponding Gaussian matrix with independent entries above the diagonal. Jiang and Li (2015) and Bubeck et al. (2016) showed that these two matrix ensembles converge in total variation whenever $d/n^3\to \infty$, and Bubeck et al. (2016) showed this to be sharp. In this paper we aim to identify the precise threshold for $d$ in terms of $n$ for subsets of Wishart matrices to converge in total variation to independent Gaussi…
