We present randomized algorithms based on block Krylov space method for estimating the trace and log-determinant of Hermitian positive semi-definite matrices. Using the properties of Chebyshev polynomial and Gaussian random matrix, we provide the error analysis of the proposed estimators and obtain the expectation and concentration error bounds. These bounds improve the corresponding ones given in the literature. Numerical experiments are presented to illustrate the performance of the algorithms and to test the error bounds.
Randomized orthogonal projection methods (ROPMs) can be used to speed up the computation of Krylov subspace methods in various contexts. Through a theoretical and numerical investigation, we establish that these methods produce quasi-optimal approximations over the Krylov subspace. Our numerical experiments outline the convergence of ROPMs for all matrices in our test set, with occasional spikes, but overall with a convergence rate similar to that of standard OPMs.
Randomized block Krylov subspace methods form a powerful class of algorithms for computing the extreme eigenvalues of a symmetric matrix or the extreme singular values of a general matrix. The purpose of this paper is to develop new theoretical bounds on the performance of randomized block Krylov subspace methods for these problems. For matrices with polynomial spectral decay, the randomized block Krylov method can obtain an accurate spectral norm estimate using only a constant number of steps (that depends on the decay rate and the accuracy). Furthermore, the analysis reveals that the behavio…
Randomized block Krylov method for approximation of truncated tensor SVD
Research Center of the Artificial Intelligence Institute, and Lab of Machine Learning and Knowledge Representation, Innopolis University（因诺波利斯大学人工智能研究所研究中心及机器学习和知识表示实验室）；Department of Teleinformatics Engineering, Federal University of Ceara（福塔莱萨塞阿拉联邦大学电信信息工程系）
The computation of f(A)b, the action of a matrix function on a vector, is a task arising in many areas of scientific computing. In many applications, the matrix A is sparse but so large that only a rather small number of Krylov basis vectors can be stored. Here we discuss a new approach to overcome these limitations by randomized sketching combined with an integral representation of f(A)b. Two different approximations are introduced, one based on sketched FOM and another based on sketched GMRES approximation. The convergence of the latter method is analyzed for Stieltjes functions of positive …
Guangdong University of Technology（广东工业大学）；RIKEN（理化学研究所）
Efficient and accurate low-rank approximation (LRA) methods are of great significance for large-scale data analysis. Randomized tensor decompositions have emerged as powerful tools to meet this need, but most existing methods perform poorly in the presence of noise interference. Inspired by the remarkable performance of randomized block Krylov iteration (rBKI) in reducing the effect of tail singular values, this work designs an rBKI-based Tucker decomposition (rBKI-TK) for accurate approximation, together with a hierarchical tensor ring decomposition based on rBKI-TK for efficient compression …
Accelerating a restarted Krylov method for matrix functions with randomization
INESC-ID, Instituto Superior Técnico, Universidade de Lisboa（葡萄牙里斯本大学高等技术学院）；Department of Mathematics and Oden Institute, University of Texas at Austin（美国德克萨斯大学奥斯汀分校数学系和奥登研究所）；Department of Mathematics, Carlos III University of Madrid（西班牙马德里卡洛斯三世大学数学系）
Regularization Properties of the Krylov Iterative Solvers CGME and LSMR For Linear Discrete Ill-Posed Problems with an Application to Truncated Randomized SVDs
Randomized trace estimation is a popular and well studied technique that approximates the trace of a large-scale matrix $B$ by computing the average of $x^T Bx$ for many samples of a random vector $X$. Often, $B$ is symmetric positive definite (SPD) but a number of applications give rise to indefinite $B$. Most notably, this is the case for log-determinant estimation, a task that features prominently in statistical learning, for instance in maximum likelihood estimation for Gaussian process regression. The analysis of randomized trace estimates, including tail bounds, has mostly focused on the…
This survey describes probabilistic algorithms for linear algebra computations, such as factorizing matrices and solving linear systems. It focuses on techniques that have a proven track record for real-world problem instances. The paper treats both the theoretical foundations of the subject and the practical computational issues. Topics covered include norm estimation; matrix approximation by sampling; structured and unstructured random embeddings; linear regression problems; low-rank approximation; subspace iteration and Krylov methods; error estimation and adaptivity; interpolatory and CUR …
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；University of Chinese Academy of Sciences（中国科学院大学）
We compared the regular Singular Value Decomposition (SVD), truncated SVD, Krylov method and Randomized PCA, in terms of time and space complexity. It is well-known that Krylov method and Randomized PCA only performs well when k << n, i.e. the number of eigenpair needed is far less than that of matrix size. We compared them for calculating all the eigenpairs. We also discussed the relationship between Principal Component Analysis and SVD.
Pass-efficient Randomized Algorithms for Low-rank Approximation of Quaternion Matrices
Innopolis University（伊诺波利斯大学）；Research Center of the Artificial Intelligence Institute, and Lab of Machine Learning and Knowledge Representation, Innopolis University（伊诺波利斯大学人工智能研究所研究中心及机器学习与知识表示实验室）
Novel technique based on Léja Points Approximation for Log-determinant Estimation of Large matrices
The University of Dodoma（多多马大学）；African Institute for Mathematical Sciences, Research and Innovation Centre (AIMSRIC)（非洲数学科学研究所研究与创新中心）；Western Norway University of Applied Sciences（挪威西部应用科学大学）；University of Cape Town（开普敦大学）
ETH Zurich（苏黎世联邦理工学院）；University of California, Berkeley（加州大学伯克利分校）
We analyze the convergence rate of the randomized Newton-like method introduced by Qu et. al. (2016) for smooth and convex objectives, which uses random coordinate blocks of a Hessian-over-approximation matrix $\bM$ instead of the true Hessian. The convergence analysis of the algorithm is challenging because of its complex dependence on the structure of $\bM$. However, we show that when the coordinate blocks are sampled with probability proportional to their determinant, the convergence rate depends solely on the eigenvalue distribution of matrix $\bM$, and has an analytically tractable form. …
A Range-Deflated Stochastic Lanczos Quadrature Method for Large-Scale Log-Determinant Estimation
The University of Dodoma（多多马大学）；African Institute for Mathematical Sciences, Research and Innovation Centre（非洲数学科学研究所研究与创新中心）；Western Norway University of Applied Sciences（挪威西部应用科学大学）；University of Cape Town（开普敦大学）
This article presents a randomized matrix-free method for approximating the trace of $f({\bf A})$, where ${\bf A}$ is a large symmetric matrix and $f$ is a function analytic in a closed interval containing the eigenvalues of ${\bf A}$. Our method uses a combination of stochastic trace estimation (i.e., Hutchinson's method), Chebyshev approximation, and multilevel Monte Carlo techniques. We establish general bounds on the approximation error of this method by extending an existing error bound for Hutchinson's method to multilevel trace estimators. Numerical experiments are conducted for common …
Convergence of the Cumulant Expansion and Polynomial-Time Algorithm for Weakly Interacting Fermions
Stanford University（斯坦福大学）；Inria, Télécom Paris - LTCI, Institut Polytechnique de Paris（法国国家信息与自动化研究所，巴黎电信学院-通信与图像处理实验室，巴黎理工学院）；California Institute of Technology（加州理工学院）；Institute for Quantum Information and Matter（量子信息物质研究所）；Simons Institute for the Theory of Computing, University of California, Berkeley（加州大学伯克利分校西蒙斯计算理论研究所）；University of California, Davis（加州大学戴维斯分校）；Department of Mathematics, Stanford University（斯坦福大学数学系）；Institute for Computational and Mathematical Engineering, Stanford University（斯坦福大学计算与数学工程研究所）；Duke University（杜克大学）；Department of Electrical and Computer Engineering, Duke University（杜克大学电气与计算机工程系）；Duke Quantum Center, Duke University（杜克大学杜克量子中心）
