This paper develops nonasymptotic information inequalities for the estimation of the eigenspaces of a covariance operator. These results generalize previous lower bounds for the spiked covariance model, and they show that recent upper bounds for models with decaying eigenvalues are sharp. The proof relies on lower bound techniques based on group invariance arguments which can also deal with a variety of other statistical models.
The paper demonstrates the application of statistical based methodology for the analysis of the vertical deviation angle. The studied data set contains astro-geodetic observations. The Principal Component Analysis and the Multiple Linear Regression models are embedded within a bootstrap procedure, in order to overcome the difficulties related to data correlation, while taking advantage of all the information provided. The methodology is applied on real data. The obtained results indicate that the pressure, the temperature and the humidity are variables that may influence the measure of the ver…
Principal Moment Analysis is a method designed for dimension reduction, analysis and visualization of high dimensional multivariate data. It generalizes Principal Component Analysis and allows for significant statistical modeling flexibility, when approximating an unknown underlying probability distribution, by enabling direct analysis of general approximate measures. Through https://principalmomentanalysis.github.io/ we provide an implementation, together with a graphical user interface, of a simplex based version of Principal Moment Analysis.
In recent years, sparse principal component analysis has emerged as an extremely popular dimension reduction technique for high-dimensional data. The theoretical challenge, in the simplest case, is to estimate the leading eigenvector of a population covariance matrix under the assumption that this eigenvector is sparse. An impressive range of estimators have been proposed; some of these are fast to compute, while others are known to achieve the minimax optimal rate over certain Gaussian or sub-Gaussian classes. In this paper, we show that, under a widely-believed assumption from computational …
National University of Singapore（新加坡国立大学）；Academy of Mathematics and Systems Science, Chinese Academy of Sciences（中国科学院数学与系统科学研究院）；Tencent（腾讯）
In this paper, we study the problem of principal component analysis with generative modeling assumptions, adopting a general model for the observed matrix that encompasses notable special cases, including spiked matrix recovery and phase retrieval. The key assumption is that the underlying signal lies near the range of an $L$-Lipschitz continuous generative model with bounded $k$-dimensional inputs. We propose a quadratic estimator, and show that it enjoys a statistical rate of order $\sqrt{\frac{k\log L}{m}}$, where $m$ is the number of samples. We also provide a near-matching algorithm-indep…
Fan et al. [$\mathit{Annals}$ $\mathit{of}$ $\mathit{Statistics}$ $\textbf{47}$(6) (2019) 3009-3031] constructed a distributed principal component analysis (PCA) algorithm to reduce the communication cost between multiple servers significantly. However, their algorithm's guarantee is only for sub-Gaussian data. Spurred by this deficiency, this paper enhances the effectiveness of their distributed PCA algorithm by utilizing robust covariance matrix estimators of Minsker [$\mathit{Annals}$ $\mathit{of}$ $\mathit{Statistics}$ $\textbf{46}$(6A) (2018) 2871-2903] and Ke et al. [$\mathit{Statistical…
Outliers contaminating data sets are a challenge to statistical estimators. Even a small fraction of outlying observations can heavily influence most classical statistical methods. In this paper we propose generalized spherical principal component analysis, a new robust version of principal component analysis that is based on the generalized spatial sign covariance matrix. Supporting theoretical properties of the proposed method including influence functions, breakdown values and asymptotic efficiencies are studied, and a simulation study is conducted to compare our new method to existing meth…
In the course of the last century, Principal Component Analysis (PCA) have become one of the pillars of modern scientific methods. Although PCA is normally addressed as a statistical tool aiming at finding orthogonal directions on which the variance is maximized, its first introduction by Pearson at 1901 was done through defining a non-linear least-squares minimization problem of fitting a plane to scattered data points. Thus, it seems natural that PCA and linear least-squares regression are somewhat related, as they both aim at fitting planes to data points. In this paper, we present a connec…
This tutorial reviews the main steps of the principal component analysis of a multivariate data set and its subsequent dimensional reduction on the grounds of identified dominant principal components. The underlying computations are demonstrated and performed by means of a script written in the statistical software package R.
In this work, we develop a novel principal component analysis (PCA) for semimartingales by introducing a suitable spectral analysis for the quadratic variation operator. Motivated by high-dimensional complex systems typically found in interest rate markets, we investigate correlation in high-dimensional high-frequency data generated by continuous semimartingales. In contrast to the traditional PCA methodology, the directions of large variations are not deterministic, but rather they are bounded variation adapted processes which maximize quadratic variation almost surely. This allows us to redu…
We introduce a new method for sparse principal component analysis, based on the aggregation of eigenvector information from carefully-selected axis-aligned random projections of the sample covariance matrix. Unlike most alternative approaches, our algorithm is non-iterative, so is not vulnerable to a bad choice of initialisation. We provide theoretical guarantees under which our principal subspace estimator can attain the minimax optimal rate of convergence in polynomial time. In addition, our theory provides a more refined understanding of the statistical and computational trade-off in the pr…
The first order behavior of multivariate heavy-tailed random vectors above large radial thresholds is ruled by a limit measure in a regular variation framework. For a high dimensional vector, a reasonable assumption is that the support of this measure is concentrated on a lower dimensional subspace, meaning that certain linear combinations of the components are much likelier to be large than others. Identifying this subspace and thus reducing the dimension will facilitate a refined statistical analysis. In this work we apply Principal Component Analysis (PCA) to a re-scaled version of radially…
New York University（纽约大学）；Princeton University（普林斯顿大学）；University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
The growing size of modern data sets brings many challenges to the existing statistical estimation approaches, which calls for new distributed methodologies. This paper studies distributed estimation for a fundamental statistical machine learning problem, principal component analysis (PCA). Despite the massive literature on top eigenvector estimation, much less is presented for the top-$L$-dim ($L>1$) eigenspace estimation, especially in a distributed manner. We propose a novel multi-round algorithm for constructing top-$L$-dim eigenspace for distributed data. Our algorithm takes advantage of …
We present a novel technique for sparse principal component analysis. This method, named Eigenvectors from Eigenvalues Sparse Principal Component Analysis (EESPCA), is based on the formula for computing squared eigenvector loadings of a Hermitian matrix from the eigenvalues of the full matrix and associated sub-matrices. We explore two versions of the EESPCA method: a version that uses a fixed threshold for inducing sparsity and a version that selects the threshold via cross-validation. Relative to the state-of-the-art sparse PCA methods of Witten et al., Yuan & Zhang and Tan et al., the fixed…
The release of NBA player tracking data greatly enhances the granularity and dimensionality of basketball statistics used to evaluate and compare player performance. However, the high dimensionality of this new data source can be troublesome as it demands more computational resources and reduces the ability to easily interpret findings. Therefore, we must find a way to reduce the dimensionality of the data while retaining the ability to differentiate and compare player performance. In this paper, Principal Component Analysis (PCA) is used to identify four principal components that account for …
Another Look at Log-PCA for Probability Measures: A Dynamical Formulation and Statistical Convergence
Department of Statistics University of Illinois Urbana-Champaign（统计学系伊利诺伊大学厄巴纳-香槟分校）；Department of ACMS University of Notre Dame（ACMS系诺丁汉大学）；Department of Mathematics University of British Columbia（数学系不列颠哥伦比亚大学）；Department of Mathematics Thomas Lord Department of Computer Science University of Southern California（数学系托马斯·劳德计算机科学系南加州大学）
The principal support vector machines method (Li et al., 2011) is a powerful tool for sufficient dimension reduction that replaces original predictors with their low-dimensional linear combinations without loss of information. However, the computational burden of the principal support vector machines method constrains its use for massive data. To address this issue, we in this paper propose two distributed estimation algorithms for fast implementation when the sample size is large. Both the two distributed sufficient dimension reduction estimators enjoy the same statistical efficiency as mergi…
In this paper, we study the asymptotic behavior of the extreme eigenvalues and eigenvectors of the high dimensional spiked sample covariance matrices, in the supercritical case when a reliable detection of spikes is possible. Especially, we derive the joint distribution of the extreme eigenvalues and the generalized components of the associated eigenvectors, i.e., the projections of the eigenvectors onto arbitrary given direction, assuming that the dimension and sample size are comparably large. In general, the joint distribution is given in terms of linear combinations of finitely many Gaussi…
This study derives a new property of the Wishart distribution when the degree-of-freedom and the size of the matrix parameter of the distribution grow simultaneoulsy. Particularly, the asymptotic normality of the product of four independent Wishart matrices is shown under a high dimensional asymptotic regime. As an application of the result, a statistical test procedure for the common principal components hypothesis is proposed. For this problem, the proposed test statistic is asymptotically normal under the null hypothesis. In addition, the proposed test statistic diverges to positive infinit…
Principal Components Regression (PCR) is a traditional tool for dimension reduction in linear regression that has been both criticized and defended. One concern about PCR is that obtaining the leading principal components tends to be computationally demanding for large data sets. While random projections do not possess the optimality properties of the leading principal subspace, they are computationally appealing and hence have become increasingly popular in recent years. In this paper, we present an analysis showing that for random projections satisfying a Johnson-Lindenstrauss embedding prop…
