Randomized sampling has recently been proven a highly efficient technique for computing approximate factorizations of matrices that have low numerical rank. This paper describes an extension of such techniques to a wider class of matrices that are not themselves rank-deficient, but have off-diagonal blocks that are; specifically, the classes of so called \textit{Hierarchically Off-Diagonal Low Rank (HODLR)} matrices and \textit{Hierarchically Block Separable (HBS)} matrices. Such matrices arise frequently in numerical analysis and signal processing, in particular in the construction of fast methods for solving differential and integral equations numerically. These structures admit algebraic operations (matrix-vector multiplications, matrix factorizations, matrix inversion, etc.) to be performed very rapidly; but only once a data-sparse representation of the matrix has been constructed. How to rapidly compute this representation in the first place is much less well understood. The present paper demonstrates that if an $N\times N$ matrix can be applied to a vector in $O(N)$ time, and if the ranks of the off-diagonal blocks are bounded by an integer $k$, then the cost for constructing a HODLR representation is $O(k^{2}\,N\,(\log N)^{2})$, and the cost for constructing an HBS representation is $O(k^{2}\,N\,\log N)$. The point is that when legacy codes (based on, e.g., the Fast Multipole Method) can be used for the fast matrix-vector multiply, the proposed algorithm can be used to obtain the data-sparse representation of the matrix, and then well-established techniques for HODLR/HBS matrices can be used to invert or factor the matrix. The proposed scheme is also useful in simplifying the implementation of certain operations on rank-structured matrices such as the matrix-matrix multiplication, low-rank update, addition, etc.
This survey describes probabilistic algorithms for linear algebra computations, such as factorizing matrices and solving linear systems. It focuses on techniques that have a proven track record for real-world problem instances. The paper treats both the theoretical foundations of the subject and the practical computational issues. Topics covered include norm estimation; matrix approximation by sampling; structured and unstructured random embeddings; linear regression problems; low-rank approximation; subspace iteration and Krylov methods; error estimation and adaptivity; interpolatory and CUR …
Simon Fraser University（西蒙弗雷泽大学）；Microsoft Research（微软研究院）
We propose a randomized nonmonotone block proximal gradient (RNBPG) method for minimizing the sum of a smooth (possibly nonconvex) function and a block-separable (possibly nonconvex nonsmooth) function. At each iteration, this method randomly picks a block according to any prescribed probability distribution and solves typically several associated proximal subproblems that usually have a closed-form solution, until a certain progress on objective value is achieved. In contrast to the usual randomized block coordinate descent method [23,20], our method has a nonmonotone flavor and uses variable…
We propose an unconditionally robust and highly effective preconditioner for general symmetric positive definite (SPD) matrices based on structured incomplete factorization (SIF), called enhanced SIF (eSIF) preconditioner. The original SIF strategy proposed recently derives a structured preconditioner by applying block diagonal preprocessing to the matrix and then compressing appropriate scaled off-diagonal blocks. Here, we use an enhanced scaling-and-compression strategy to design the new eSIF preconditioner. Some subtle modifications are made, such as the use of two-sided block triangular pr…
A random matrix is likely to be well conditioned, and motivated by this well known property we employ random matrix multipliers to advance some fundamental matrix computations. This includes numerical stabilization of Gaussian elimination with no pivoting as well as block Gaussian elimination, approximation of the leading and trailing singular spaces of an ill conditioned matrix, associated with its largest and smallest singular values, respectively, and approximation of this matrix by low-rank matrices, with further extensions to computing numerical ranks and the approximation of tensor decom…
Focusing inversion of potential field data for the recovery of sparse subsurface structures from surface measurement data on a uniform grid is discussed. For the uniform grid the model sensitivity matrices exhibit block Toeplitz Toeplitz block structure, by blocks for each depth layer of the subsurface. Then, through embedding in circulant matrices, all forward operations with the sensitivity matrix, or its transpose, are realized using the fast two dimensional Fourier transform. Simulations demonstrate that this fast inversion algorithm can be implemented on standard desktop computers with su…
