In this paper, we consider approximations of principal component projection (PCP) without explicitly computing principal components. This problem has been studied in several recent works. The main feature of existing approaches is viewing the PCP matrix as a matrix function. This underlying function is the composition of a step function with a rational function. To find an approximate PCP, the step function is approximated by a polynomial while the rational function is evaluated by a fast ridge regression solver. In this work, we further improve this process by replacing the rational function with carefully constructed polynomials of low degree. We characterize the properties of polynomials that are suitable for approximating PCP, and establish an optimization problem to select the optimal one from those polynomials. We show theoretically and confirm numerically that the resulting approximate PCP approach with optimal polynomials is indeed effective for approximations of principal component projection.
A real square matrix is Perron-like if it has a real eigenvalue $s$, called the principal eigenvalue of the matrix, and $\mbox{Re}\,μ<s$ for any other eigenvalue $μ$. Nonnegative matrices and symmetric ones are typical examples of this class of matrices. The main purpose of this paper is to develop a set of new schemes to compute the principal eigenvalues of Perron-like matrices and the associated generalized eigenspaces by using polynomial approximations of matrix exponentials. Numerical examples show that these schemes are effective in practice.
Accelerated Alternating Projections for Robust Principal Component Analysis
Department of Mathematics, University of California, Los Angeles（加州大学洛杉矶分校数学系）；Department of Mathematics, Hong Kong University of Science and Technology（香港理工大学数学系）；School of Data Science, Fudan University（复旦大学数据科学学院）
We compare the convergence behavior of best polynomial approximations and Legendre and Chebyshev projections and derive optimal rates of convergence of Legendre projections for analytic and differentiable functions in the maximum norm. For analytic functions, we show that the best polynomial approximation of degree $n$ is better than the Legendre projection of the same degree by a factor of $n^{1/2}$. For differentiable functions such as piecewise analytic functions and functions of fractional smoothness, however, we show that the best approximation is better than the Legendre projection by on…
The Galois group of a parameterized polynomial system of equations encodes the structure of the solutions. This monodromy group acts on the set of solutions for a general set of parameters, that is, on the fiber of a projection from the incidence variety of parameters and solutions onto the space of parameters. When this projection is decomposable, the Galois group is imprimitive, and we show that the structure can be exploited for computational improvements. Furthermore, we develop a new algorithm for solving these systems based on a suitable trace test. We illustrate our method on examples i…
We show stability of the $L^2$-projection onto Lagrange finite element spaces with respect to (weighted) $L^p$ and $W^{1,p}$-norms for any polynomial degree and for any space dimension under suitable conditions on the mesh grading. This includes $W^{1,2}$-stability in two space dimensions for any polynomial degree and meshes generated by newest vertex bisection. Under realistic but conjectured assumptions on the mesh grading in three dimensions we show $W^{1,2}$-stability for all polynomial degrees. We also propose a modified bisection strategy that leads to better $W^{1,p}$-stability. Moreove…
The main purpose of our paper is a new approach to design of algorithms of Kaczmarz type in the framework of operators in Hilbert space. Our applications include a diverse list of optimization problems, new Karhunen-Loève transforms, and Principal Component Analysis (PCA) for digital images. A key feature of our algorithms is our use of recursive systems of projection operators. Specifically, we apply our recursive projection algorithms for new computations of PCA probabilities and of variance data. For this we also make use of specific reproducing kernel Hilbert spaces, factorization for kern…
The present work addresses the issue of accurate stochastic approximations in high-dimensional parametric space using tools from uncertainty quantification (UQ). The basis adaptation method and its accelerated algorithm in polynomial chaos expansions (PCE) were recently proposed to construct low-dimensional approximations adapted to specific quantities of interest (QoI). The present paper addresses one difficulty with these adaptations, namely their reliance on quadrature point sampling, which limits the reusability of potentially expensive samples. Projection pursuit (PP) is a statistical too…
