For a given matrix, we are interested in computing GR decompositions $A=GR$, where $G$ is an isometry with respect to given scalar products. The orthogonal QR decomposition is the representative for the Euclidian scalar product. For a signature matrix, a respective factorization is given as the hyperbolic QR decomposition. Considering a skew-symmetric matrix leads to the symplectic QR decomposition. The standard approach for computing GR decompositions is based on the successive elimination of subdiagonal matrix entries. For the hyperbolic and symplectic case, this approach does in general not lead to a satisfying numerical accuracy. An alternative approach computes the QR decomposition via a Cholesky factorization, but also has bad stability. It is improved by repeating the procedure a second time. In the same way, the hyperbolic and the symplectic QR decomposition are related to the $LDL^T$ and a skew-symmetric Cholesky-like factorization. We show that methods exploiting this connection can provide better numerical stability than elimination-based approaches.
This article proposes and analyzes several variants of the randomized Cholesky QR factorization of a matrix $X$. Instead of computing the R factor from $X^T X$, as is done by standard methods, we obtain it from a small, efficiently computable random sketch of $X$, thus saving computational cost and improving numerical stability. The proposed direct variant of the randomized Cholesky QR requires only half the flops and the same communication cost as the classical Cholesky QR. At the same time, it is more robust since it is guaranteed to be stable whenever the input matrix is numerically full-ra…
Shifted CholeskyQR for computing the QR factorization of ill-conditioned matrices
Hokkaido University（北海道大学）；Arup（奥雅纳）；National Institute of Informatics（国立情报学研究所）；The University of Electro-Communications（电气通信大学）；JST CREST（日本科学技术振兴机构战略创造事业）；Waseda university（早稻田大学）
We introduce a randomized algorithm, namely RCHOL, to construct an approximate Cholesky factorization for a given Laplacian matrix (a.k.a., graph Laplacian). From a graph perspective, the exact Cholesky factorization introduces a clique in the underlying graph after eliminating a row/column. By randomization, RCHOL only retains a sparse subset of the edges in the clique using a random sampling developed by Spielman and Kyng. We prove RCHOL is breakdown-free and apply it to solving large sparse linear systems with symmetric diagonally dominant matrices. In addition, we parallelize RCHOL based o…
Large kernel systems are prone to be ill-conditioned. Pivoted Cholesky decomposition (PCD) render a stable and efficient solution to the systems without a perturbation of regularization. This paper proposes a new PCD algorithm by tuning Cross Approximation (CA) algorithm to kernel matrices which merges the merits of PCD and CA, and proves as well as numerically exemplifies that it solves large kernel systems two-order more efficiently than those resorts to regularization. As a by-product, a diagonal-pivoted CA technique is also shown efficient in eigen-decomposition of large covariance matrice…
A new algorithm to approximate Hermitian matrices by positive semidefinite Hermitian matrices based on modified Cholesky decompositions is presented. In contrast to existing algorithms, this algorithm allows to specify bounds on the diagonal values of the approximation. It has no significant runtime and memory overhead compared to the computation of a classical Cholesky decomposition. Hence it is suitable for large matrices as well as sparse matrices since it preserves the sparsity pattern of the original matrix. The algorithm tries to minimize the approximation error in the Frobenius norm as …
We analyze two algorithms for computing the symplectic $LL^T$ factorization $A=LL^T$ of a given symmetric positive definite symplectic matrix $A$. The first algorithm $W_1$ is an implementation of the $HH^T$ factorization from [Dopico et al., 2009], see Theorem 5.2. The second one, algorithm $W_2$ uses both Cholesky and Reverse Cholesky decompositions of symmetric positive definite matrices. We presents a comparison of these algorithms and illustrate their properties by numerical experiments in MATLAB. A particular emphasis is given on simplecticity properties of the computed matrices in float…
Mixed precision multigrid with smoothing based on incomplete Cholesky factorization
CNRS-IRIT（法国国家科学研究中心-信息学与系统技术研究所）；IFP Energies Nouvelles（法国石油研究院）；Department of Numerical Mathematics, Charles University（布拉格查理大学数值数学系）；Technical University of Munich - Campus Heilbronn（慕尼黑工业大学海尔布隆校区）；Ludwig-Maximilians-Universität München - Department of Earth and Environmental Sciences（慕尼黑路德维希-马克西米利安大学地球与环境科学系）；Department of Computer Science, FAU Erlangen-Nürnberg（埃尔朗根-纽伦堡大学计算机系）；Department of Applied Mathematics, VSB-Technical University of Ostrava（俄斯特拉发理工大学应用数学系）；Centre Européen de Recherche et de Formation Avancée en Calcul Scientifique（欧洲高级科学计算研究与培训中心）
Greville's method has been utilized in (Broad Learn-ing System) BLS to propose an effective and efficient incremental learning system without retraining the whole network from the beginning. For a column-partitioned matrix where the second part consists of p columns, Greville's method requires p iterations to compute the pseudoinverse of the whole matrix from the pseudoinverse of the first part. The incremental algorithms in BLS extend Greville's method to compute the pseudoinverse of the whole matrix from the pseudoinverse of the first part by just 1 iteration, which have neglected some possi…
We propose to compute a sparse approximate inverse Cholesky factor $L$ of a dense covariance matrix $Θ$ by minimizing the Kullback-Leibler divergence between the Gaussian distributions $\mathcal{N}(0, Θ)$ and $\mathcal{N}(0, L^{-\top} L^{-1})$, subject to a sparsity constraint. Surprisingly, this problem has a closed-form solution that can be computed efficiently, recovering the popular Vecchia approximation in spatial statistics. Based on recent results on the approximate sparsity of inverse Cholesky factors of $Θ$ obtained from pairwise evaluation of Green's functions of elliptic boundary-va…
We present methods for computing the generalized polar decomposition of a matrix based on the dynamically weighted Halley (DWH) iteration. This method is well established for computing the standard polar decomposition. A stable implementation is available, where matrix inversion is avoided and QR decompositions are used instead. We establish a natural generalization of this approach for computing generalized polar decompositions with respect to signature matrices. Again the inverse can be avoided by using a generalized QR decomposition called hyperbolic QR decomposition. However, this decompos…
We consider the problem of writing an arbitrary symmetric matrix as the difference of two positive semidefinite matrices. We start with simple ideas such as eigenvalue decomposition. Then, we develop a simple adaptation of the Cholesky that returns a difference-of-Cholesky representation of indefinite matrices. Heuristics that promote sparsity can be applied directly to this modification.
The success of matrix factorizations such as the singular value decomposition (SVD) has motivated the search for even more factorizations. We catalog 53 matrix factorizations, most of which we believe to be new. Our systematic approach, inspired by the generalized Cartan decomposition of Lie theory, also encompasses known factorizations such as the SVD, the symmetric eigendecomposition, the CS decomposition, the hyperbolic SVD, structured SVDs, the Takagi factorization, and others thereby covering familiar matrix factorizations as well as ones that were waiting to be discovered. We suggest tha…
LU and Cholesky matrix factorization algorithms are core subroutines used to solve systems of linear equations (SLEs) encountered while solving an optimization problem. Standard factorization algorithms are highly efficient but remain susceptible to the accumulation of roundoff errors, which can lead solvers to return feasibility and optimality claims that are actually invalid. This paper introduces a novel approach for solving sequences of closely related SLEs encountered in nonlinear programming efficiently and without roundoff errors. Specifically, it introduces rank-one update algorithms f…
