Many application problems that lead to solving linear systems make use of preconditioned Krylov subspace solvers to compute their solution. Among the most popular preconditioning approaches are incomplete factorization methods either as single-level approaches or within a multilevel framework. We will present a block incomplete factorization that is based on skillfully blocking the system initially and throughout the factorization. This approach allows for the use of cache-optimized dense matrix kernels such as level-3 BLAS or LAPACK. We will demonstrate how this block approach outperforms the scalar method often by orders of magnitude on modern architectures, paving the way for its prospective use inside various multilevel incomplete factorization approaches or other applications where the core part relies on an incomplete factorization.
SYM-ILDL is a numerical software package that computes incomplete $LDL^{T}$ (or `ILDL') factorizations of symmetric indefinite and real skew-symmetric matrices. The core of the algorithm is a Crout variant of incomplete LU (ILU), originally introduced and implemented for symmetric matrices by [Li and Saad, Crout versions of ILU factorization with pivoting for sparse symmetric matrices, Transactions on Numerical Analysis 20, pp. 75--85, 2005]. Our code is economical in terms of storage and it deals with real skew-symmetric matrices as well, in addition to symmetric ones. The package is written …
We introduce a software package called HIFIR for preconditioning sparse, unsymmetric, ill-conditioned, and potentially singular systems. HIFIR computes a hybrid incomplete factorization, which combines multilevel incomplete LU factorization with a truncated, rank-revealing QR factorization on the final Schur complement. This novel hybridization is based on the new theory of approximate generalized inverse and $ε$-accuracy. It enables near-optimal preconditioners for consistent systems and enables flexible GMRES to solve inconsistent systems when coupled with iterative refinement. In this paper…
A hybrid recursive multilevel incomplete factorization preconditioner for solving general linear systems
Johann Bernoulli Institute for Mathematics and Computer Science - University of Groningen（格罗宁根大学）；School of Mathematical Sciences, University of Electronic Science and Technology of China（电子科技大学）
We consider using the preconditioned-Krylov subspace method to solve the system of linear equations with a three-by-three block structure. By making use of the three-by-three block structure, eight inexact block factorization preconditioners, which can be put into a same theoretical analysis frame, are proposed based on a kind of inexact factorization. By generalizing Bendixson Theorem and developing a unified technique of spectral equivalence, the bounds of the real and imaginary parts of eigenvalues of the preconditioned matrices are obtained. The comparison to eleven existed exact and inexa…
Mixed precision multigrid with smoothing based on incomplete Cholesky factorization
CNRS-IRIT（法国国家科学研究中心-信息学与系统技术研究所）；IFP Energies Nouvelles（法国石油研究院）；Department of Numerical Mathematics, Charles University（布拉格查理大学数值数学系）；Technical University of Munich - Campus Heilbronn（慕尼黑工业大学海尔布隆校区）；Ludwig-Maximilians-Universität München - Department of Earth and Environmental Sciences（慕尼黑路德维希-马克西米利安大学地球与环境科学系）；Department of Computer Science, FAU Erlangen-Nürnberg（埃尔朗根-纽伦堡大学计算机系）；Department of Applied Mathematics, VSB-Technical University of Ostrava（俄斯特拉发理工大学应用数学系）；Centre Européen de Recherche et de Formation Avancée en Calcul Scientifique（欧洲高级科学计算研究与培训中心）
We present two new algorithms for Householder QR factorization of Block Low-Rank (BLR) matrices: one that performs block-column-wise QR, and another that is based on tiled QR. We show how the block-column-wise algorithm exploits BLR structure to achieve arithmetic complexity of $\mathcal{O}(mn)$, while the tiled BLR-QR exhibits $\mathcal{O}(mn^{1.5})$ complexity. However, the tiled BLR-QR has finer task granularity that allows parallel task-based execution on shared memory systems. We compare the block-column-wise BLR-QR using fork-join parallelism with tiled BLR-QR using task-based parallelis…
The problem of incomplete data - i.e., data with missing or unknown values - in multi-way arrays is ubiquitous in biomedical signal processing, network traffic analysis, bibliometrics, social network analysis, chemometrics, computer vision, communication networks, etc. We consider the problem of how to factorize data sets with missing values with the goal of capturing the underlying latent structure of the data and possibly reconstructing missing values (i.e., tensor completion). We focus on one of the most well-known tensor factorizations that captures multi-linear structure, CANDECOMP/PARAFA…
We propose BIBPA, a block inertial Bregman proximal algorithm for minimizing the sum of a block relatively smooth function (that is, relatively smooth concerning each block) and block separable nonsmooth nonconvex functions. We prove that the sequence generated by BIBPA subsequentially converges to critical points of the objective under standard assumptions, and globally converges when the objective function is additionally assumed to satisfy the Kurdyka-Łojasiewicz (KŁ) property. We also provide the convergence rate when the objective satisfies the Łojasiewicz inequality. We apply BIBPA to th…
Matrix factorization techniques compute low-rank product approximations of high dimensional data matrices and as a result, are often employed in recommender systems and collaborative filtering applications. However, many algorithms for this task utilize an exact least-squares solver whose computation is time consuming and memory-expensive. In this paper we discuss and test a block Kaczmarz solver that replaces the least-squares subroutine in the common alternating scheme for matrix factorization. This variant trades a small increase in factorization error for significantly faster algorithmic p…
Université de Mons（蒙斯大学）；Université de Rennes（雷恩大学）；INRIA（法国国家信息与自动化研究所）；CNRS（法国国家科学研究中心）；IRISA（雷恩第一大学计算机科学研究所）
This paper is concerned with improving the empirical convergence speed of block-coordinate descent algorithms for approximate nonnegative tensor factorization (NTF). We propose an extrapolation strategy in-between block updates, referred to as heuristic extrapolation with restarts (HER). HER significantly accelerates the empirical convergence speed of most existing block-coordinate algorithms for dense NTF, in particular for challenging computational scenarios, while requiring a negligible additional computational budget.
We introduce and analyze BPALM and A-BPALM, two multi-block proximal alternating linearized minimization algorithms using Bregman distances for solving structured nonconvex problems. The objective function is the sum of a multi-block relatively smooth function (i.e., relatively smooth by fixing all the blocks except one) and block separable (nonsmooth) nonconvex functions. It turns out that the sequences generated by our algorithms are subsequentially convergent to critical points of the objective function, while they are globally convergent under KL inequality assumption. Further, the rate of…
We present a study of the effectiveness of asynchronous incomplete LU factorization preconditioners for the time-implicit solution of compressible flow problems while exploiting thread-parallelism within a compute node. A block variant of the asynchronous fine-grain parallel preconditioner adapted to a finite volume discretization of the compressible Navier-Stokes equations on unstructured grids is presented, and convergence theory is extended to the new variant. Experimental (numerical) results on the performance of these preconditioners on inviscid and viscous laminar two-dimensional steady-…
In this paper we develop algorithms for orthogonal similarity transformations of skew-symmetric matrices to simpler forms. The first algorithm is similar to the algorithm for the block antitriangular factorization of symmetric matrices, but in the case of skew-symmetric matrices, an antitriangular form is always obtained. Moreover, a simple two-sided permutation of the antitriangular form transforms the matrix into a multi-arrowhead matrix. In addition, we show that the block antitriangular form of the skew-Hermitian matrices has the same structure as the block antitriangular form of the symme…
Row-Splitting ILU Preconditioners for Sparse Least-Squares Problems
STFC Rutherford Appleton Laboratory（STFC卢瑟福阿普尔顿实验室）；School of Mathematical, Physical and Computational Sciences, University of Reading（雷丁大学数学、物理与计算科学学院）；Department of Numerical Mathematics, Faculty of Mathematics and Physics, Charles University（查理大学数学物理学院数值数学系）
Incomplete factorization is a widely used preconditioning technique for Krylov subspace methods for solving large-scale sparse linear systems. Its multilevel variants, such as ILUPACK, are more robust for many symmetric or unsymmetric linear systems than the traditional, single-level incomplete LU (or ILU) techniques. However, the previous multilevel ILU techniques still lacked robustness and efficiency for some large-scale saddle-point problems, which often arise from systems of partial differential equations (PDEs). We introduce HILUCSI, or Hierarchical Incomplete LU-Crout with Scalability-o…
We describe a second-order accurate approach to sparsifying the off-diagonal blocks in the hierarchical approximate factorizations of sparse symmetric positive definite matrices. The norm of the error made by the new approach depends quadratically, not linearly, on the error in the low-rank approximation of the given block. The analysis of the resulting two-level preconditioner shows that the preconditioner is second-order accurate as well. We incorporate the new approach into the recent Sparsified Nested Dissection algorithm [SIAM J. Matrix Anal. Appl., 41 (2020), pp. 715-746], and test it on…
Stanford University（斯坦福大学）；Duke University（杜克大学）；International Computer Science Institute（国际计算机科学研究所）；University of California, Berkeley（加州大学伯克利分校）
Kernel methods are widespread in machine learning; however, they are limited by the quadratic complexity of the construction, application, and storage of kernel matrices. Low-rank matrix approximation algorithms are widely used to address this problem and reduce the arithmetic and storage cost. However, we observed that for some datasets with wide intra-class variability, the optimal kernel parameter for smaller classes yields a matrix that is less well approximated by low-rank methods. In this paper, we propose an efficient structured low-rank approximation method -- the Block Basis Factoriza…
This paper presents a parallel preconditioning approach based on incomplete LU (ILU) factorizations in the framework of Domain Decomposition (DD) for general sparse linear systems. We focus on distributed memory parallel architectures, specifically, those that are equipped with graphic processing units (GPUs). In addition to block Jacobi, we present general purpose two-level ILU Schur complement-based approaches, where different strategies are presented to solve the coarse-level reduced system. These strategies are combined with modified ILU methods in the construction of the coarse-level oper…
Algorithms for studying transitions and instabilities in incompressible flows typically require the solution of linear systems with the full Jacobian matrix. Other popular approaches, like gradient-based design optimization and fully implicit time integration, also require very robust solvers for this type of linear system. We present a parallel fully coupled multilevel incomplete factorization preconditioner for the 3D stationary incompressible Navier-Stokes equations on a structured grid. The algorithm and software are based on the robust two-level method developed by Wubs and Thies. In this…
