Approximating the action of a matrix function $f(\mathbf{A})$ on a vector $\mathbf{b}$ is an increasingly important primitive in machine learning, data science, and statistics, with applications such as sampling high dimensional Gaussians, Gaussian process regression and Bayesian inference, principle component analysis, and approximating Hessian spectral densities. Over the past decade, a number of algorithms enjoying strong theoretical guarantees have been proposed for this task. Many of the most successful belong to a family of algorithms called Krylov subspace methods. Remarkably, a classic Krylov subspace method, called the Lanczos method for matrix functions (Lanczos-FA), frequently outperforms newer methods in practice. Our main result is a theoretical justification for this finding: we show that, for a natural class of rational functions, Lanczos-FA matches the error of the best possible Krylov subspace method up to a multiplicative approximation factor. The approximation factor depends on the degree of $f(x)$'s denominator and the condition number of $\mathbf{A}$, but not on the number of iterations $k$. Our result provides a strong justification for the excellent performance of Lanczos-FA, especially on functions that are well approximated by rationals, such as the matrix square root.
We analyze the Lanczos method for matrix function approximation (Lanczos-FA), an iterative algorithm for computing $f(\mathbf{A}) \mathbf{b}$ when $\mathbf{A}$ is a Hermitian matrix and $\mathbf{b}$ is a given vector. Assuming that $f : \mathbb{C} \rightarrow \mathbb{C}$ is piecewise analytic, we give a framework, based on the Cauchy integral formula, which can be used to derive a priori and a posteriori error bounds for Lanczos-FA in terms of the error of Lanczos used to solve linear systems. Unlike many error bounds for Lanczos-FA, these bounds account for fine-grained properties of the spec…
Quadratic forms of Hermitian matrix resolvents involve the solutions of shifted linear systems. Efficient iterative solutions use the shift-invariance property of Krylov subspaces The Hermitian Lanczos method reduces a given vector and matrix to a Jacobi matrix (real symmetric tridiagonal matrix with positive super and sub-diagonal entries) and approximates the quadratic form using the Jacobi matrix. This study develops a shifted Lanczos method that deals directly with the Hermitian matrix resolvent. We derive a matrix representation of a linear operator that approximates the resolvent by solv…
Stochastic Lanczos Quadrature for Computational Uncertainty in Linear Algebra
University of Dodoma（多多马大学）；African Institute for Mathematical Sciences Research and Innovation Centre（非洲数学科学研究与创新中心）；University of Tübingen（蒂宾根大学）；Tübingen AI Center（蒂宾根人工智能中心）
A Range-Deflated Stochastic Lanczos Quadrature Method for Large-Scale Log-Determinant Estimation
The University of Dodoma（多多马大学）；African Institute for Mathematical Sciences, Research and Innovation Centre（非洲数学科学研究所研究与创新中心）；Western Norway University of Applied Sciences（挪威西部应用科学大学）；University of Cape Town（开普敦大学）
A generalized skew-symmetric Lanczos bidiagonalization method for computing several extreme eigenpairs of a large skew-symmetric/symmetric positive definite matrix pair
A Thick-Restart Lanczos algorithm with polynomial filtering for Hermitian eigenvalue problems
Lawrence Livermore National Laboratory（劳伦斯利弗莫尔国家实验室）；University of Minnesota, Twin Cities（明尼苏达大学双城分校）；Lawrence Berkeley National Laboratory（劳伦斯伯克利国家实验室）
The Lanczos method is one of the standard approaches for computing a few eigenpairs of a large, sparse, symmetric matrix. It is typically used with restarting to avoid unbounded growth of memory and computational requirements. Thick-restart Lanczos is a popular restarted variant because of its simplicity and numerically robustness. However, convergence can be slow for highly clustered eigenvalues so more effective restarting techniques and the use of preconditioning is needed. In this paper, we present a thick-restart preconditioned Lanczos method, TRPL+K, that combines the power of locally op…
The cumulative empirical spectral measure (CESM) $Φ[\mathbf{A}] : \mathbb{R} \to [0,1]$ of a $n\times n$ symmetric matrix $\mathbf{A}$ is defined as the fraction of eigenvalues of $\mathbf{A}$ less than a given threshold, i.e., $Φ[\mathbf{A}](x) := \sum_{i=1}^{n} \frac{1}{n} {\large\unicode{x1D7D9}}[ λ_i[\mathbf{A}]\leq x]$. Spectral sums $\operatorname{tr}(f[\mathbf{A}])$ can be computed as the Riemann--Stieltjes integral of $f$ against $Φ[\mathbf{A}]$, so the task of estimating CESM arises frequently in a number of applications, including machine learning. We present an error analysis for st…
A thick-restart Lanczos type algorithm is proposed for Hermitian $J$-symmetric matrices. Since Hermitian $J$-symmetric matrices possess doubly degenerate spectra or doubly multiple eigenvalues with a simple relation between the degenerate eigenvectors, we can improve the convergence of the Lanczos algorithm by restricting the search space of the Krylov subspace to that spanned by one of each pair of the degenerate eigenvector pairs. We show that the Lanczos iteration is compatible with the $J$-symmetry, so that the subspace can be split into two subspaces that are orthogonal to each other. The…
Lanczos Method for QRPA Strength Functions in Atomic Nuclei
Computational Research Division, Lawrence Berkeley National Laboratory（劳伦斯伯克利国家实验室计算研究部）；Department of Physics and Astronomy, University of North Carolina（北卡罗来纳大学物理与天文学系）
Low-rank approximations of original samples are playing more and more an important role in many recently proposed mathematical models from data science. A natural and initial requirement is that these representations inherit original structures or properties. With this aim, we propose a new multi-symplectic method based on the Lanzcos bidiagonalization to compute the partial singular triplets of JRS-symmetric matrices. These singular triplets can be used to reconstruct optimal low-rank approximations while preserving the intrinsic multi-symmetry. The augmented Ritz and harmonic Ritz vectors ar…
The time-ordered exponential of a time-dependent matrix $\mathsf{A}(t)$ is defined as the function of $\mathsf{A}(t)$ that solves the first-order system of coupled linear differential equations with non-constant coefficients encoded in $\mathsf{A}(t)$. The authors recently proposed the first Lanczos-like algorithm capable of evaluating this function. This algorithm relies on inverses of time-dependent functions with respect to a non-commutative convolution-like product, denoted $\ast$. Yet, the existence of such inverses, crucial to avoid algorithmic breakdowns, still needed to be proved. Here…
For a linear matrix function $f$ in $X \in \R^{m\times n}$ we consider inhomogeneous linear matrix equations $f(X) = E$ for $E \neq 0$ that have or do not have solutions. For such systems we compute optimal norm constrained solutions iteratively using the Conjugate Gradient and Lanczos' methods in combination with the More-Sorensen optimizer. We build codes for ten linear matrix equations, of Sylvester, Lyapunov, Stein and structured types and their T-versions, that differ only in two five times repeated equation specific code lines. Numerical experiments with linear matrix equations are perfo…
We present randUBV, a randomized algorithm for matrix sketching based on the block Lanzcos bidiagonalization process. Given a matrix $\bf{A}$, it produces a low-rank approximation of the form ${\bf UBV}^T$, where $\bf{U}$ and $\bf{V}$ have orthonormal columns in exact arithmetic and $\bf{B}$ is block bidiagonal. In finite precision, the columns of both ${\bf U}$ and ${\bf V}$ will be close to orthonormal. Our algorithm is closely related to the randQB algorithms of Yu, Gu, and Li (2018) in that the entries of $\bf{B}$ are incrementally generated and the Frobenius norm approximation error may b…
Coimbra Polytechnic – ISEC（科英布拉理工学院ISEC）；Institute of Systems and Robotics, University of Coimbra（科英布拉大学系统与机器人研究所）；Department of Mathematics, Robat Karim Branch, Islamic Azad University（伊斯兰阿扎德大学罗巴特卡里姆分校数学系）
We give a novel spectral approximation algorithm for the balanced separator problem that, given a graph G, a constant balance b \in (0,1/2], and a parameter γ, either finds an Ω(b)-balanced cut of conductance O(\sqrt(γ)) in G, or outputs a certificate that all b-balanced cuts in G have conductance at least γ, and runs in time \tilde{O}(m). This settles the question of designing asymptotically optimal spectral algorithms for balanced separator. Our algorithm relies on a variant of the heat kernel random walk and requires, as a subroutine, an algorithm to compute \exp(-L)v where L is the Laplaci…
