The generalized singular value decomposition (GSVD) is a valuable tool that has many applications in computational science. However, computing the GSVD for large-scale problems is challenging. Motivated by applications in hyper-differential sensitivity analysis (HDSA), we propose new randomized algorithms for computing the GSVD which use randomized subspace iteration and weighted QR factorization. Detailed error analysis is given which provides insight into the accuracy of the algorithms and the choice of the algorithmic parameters. We demonstrate the performance of our algorithms on test matrices and a large-scale model problem where HDSA is used to study subsurface flow.
This paper presents a randomized quaternion singular value decomposition (QSVD) algorithm for low-rank matrix approximation problems, which are widely used in color face recognition, video compression, and signal processing problems. With quaternion normal distribution based random sampling, the randomized QSVD algorithm projects a high-dimensional data to a low-dimensional subspace and then identifies an approximate range subspace of the quaternion matrix. The key statistical properties of quaternion Wishart distribution are proposed and used to perform the approximation error analysis of the…
The oriented singular value decomposition (O-SVD) proposed by Zeng and Ng provides a hybrid approach to the t-product based third-order tensor singular value decomposition with the transform matrix being a factor matrix of the higher order singular value decomposition. Continuing along this vein, this paper explores realizing the O-SVD more efficiently by the tensor-train rank-1 decomposition and gives a truncated O-SVD. Motivated by the success of probabilistic algorithms, we develop a randomized version of the O-SVD and present its detailed error analysis. The new algorithm has advantages in…
In this work, we develop efficient solvers for linear inverse problems based on randomized singular value decomposition (RSVD). This is achieved by combining RSVD with classical regularization methods, e.g., truncated singular value decomposition, Tikhonov regularization, and general Tikhonov regularization with a smoothness penalty. One distinct feature of the proposed approach is that it explicitly preserves the structure of the regularized solution in the sense that it always lies in the range of a certain adjoint operator. We provide error estimates between the approximation and the exact …
The tensor Singular Value Decomposition (t-SVD) for third order tensors that was proposed by Kilmer and Martin~\cite{2011kilmer} has been applied successfully in many fields, such as computed tomography, facial recognition, and video completion. In this paper, we propose a method that extends a well-known randomized matrix method to the t-SVD. This method can produce a factorization with similar properties to the t-SVD, but is more computationally efficient on very large datasets. We present details of the algorithm, theoretical results, and provide numerical results that show the promise of o…
In this paper, we focus on developing randomized algorithms for the computation of low multilinear rank approximations of tensors based on the random projection and the singular value decomposition. Following the theory of the singular values of sub-Gaussian matrices, we make a probabilistic analysis for the error bounds for the randomized algorithm. We demonstrate the effectiveness of proposed algorithms via several numerical examples.
The randomized Gauss--Seidel method and its extension have attracted much attention recently and their convergence rates have been considered extensively. However, the convergence rates are usually determined by upper bounds, which cannot fully reflect the actual convergence. In this paper, we make a detailed analysis of their convergence behaviors. The analysis shows that the larger the singular value of $A$ is, the faster the error decays in the corresponding singular vector space, and the convergence directions are mainly driven by the large singular values at the beginning, then gradually …
Conditional value at risk (CVaR) is a popular measure for quantifying portfolio risk. Sensitivity analysis of CVaR is very useful in risk management and gradient-based optimization algorithms. In this paper, we study the infinitesimal perturbation analysis estimator for CVaR sensitivity using randomized quasi-Monte Carlo (RQMC) simulation. We first prove that the RQMC-based estimator is strongly consistent under very mild conditions. Under some technical conditions, RQMC that uses $d$-dimensional points in CVaR sensitivity estimation yields a mean error rate of $O(n^{-1/2-1/(4d-2)+ε})$ for arb…
In this paper, we analyze the convergence behavior of the randomized extended Kaczmarz (REK) method for all types of linear systems (consistent or inconsistent, overdetermined or underdetermined, full-rank or rank-deficient). The analysis shows that the larger the singular value of $A$ is, the faster the error decays in the corresponding right singular vector space, and as $k\rightarrow\infty$, $x_{k}-x_{\star}$ tends to the right singular vector corresponding to the smallest singular value of $A$, where $x_{k}$ is the $k$th approximation of the REK method and $x_{\star}$ is the minimum $\ell_…
Randomized Kaczmarz is a simple iterative method for finding solutions of linear systems $Ax = b$. We point out that the arising sequence $(x_k)_{k=1}^{\infty}$ tends to converge to the solution $x$ in an interesting way: generically, as $k \rightarrow \infty$, $x_k - x$ tends to the singular vector of $A$ corresponding to the smallest singular value. This has interesting consequences: in particular, the error analysis of Strohmer \& Vershynin is optimal. It also quantifies the `pre-convergence' phenomenon where the method initially seems to converge faster. This fact also allows for a fast co…
University of Oxford（牛津大学）；Cornell University（康奈尔大学）
The randomized singular value decomposition (SVD) is a popular and effective algorithm for computing a near-best rank $k$ approximation of a matrix $A$ using matrix-vector products with standard Gaussian vectors. Here, we generalize the randomized SVD to multivariate Gaussian vectors, allowing one to incorporate prior knowledge of $A$ into the algorithm. This enables us to explore the continuous analogue of the randomized SVD for Hilbert--Schmidt (HS) operators using operator-function products with functions drawn from a Gaussian process (GP). We then construct a new covariance kernel for GPs,…
Modified Truncated Randomized Singular Value Decomposition (MTRSVD) Algorithms for Large Scale Discrete Ill-posed Problems with General-Form Regularization
The QLP decomposition is one of the effective algorithms to approximate singular value decomposition (SVD) in numerical linear algebra. In this paper, we propose some single-pass randomized QLP decomposition algorithms for computing the low-rank matrix approximation. Compared with the deterministic QLP decomposition, the complexity of the proposed algorithms does not increase significantly and the system matrix needs to be accessed only once. Therefore, our algorithms are very suitable for a large matrix stored outside of memory or generated by stream data. In the error analysis, we give the b…
We propose a new algorithm for the computation of a singular value decomposition (SVD) low-rank approximation of a matrix in the Matrix Product Operator (MPO) format, also called the Tensor Train Matrix format. Our tensor network randomized SVD (TNrSVD) algorithm is an MPO implementation of the randomized SVD algorithm that is able to compute dominant singular values and their corresponding singular vectors. In contrast to the state-of-the-art tensor-based alternating least squares SVD (ALS-SVD) and modified alternating least squares SVD (MALS-SVD) matrix approximation methods, TNrSVD can be u…
The large-scale focusing inversion of gravity and magnetic potential field data using $L_1$-norm regularization is considered. The use of the randomized singular value decomposition methodology facilitates tackling the computational challenge that arises in the solution of these large-scale inverse problems. As such the powerful randomized singular value decomposition is used for the numerical solution of all linear systems required in the algorithm. A comprehensive comparison of the developed methodology for the inversion of magnetic and gravity data is presented. These results indicate that …
RSVDPACK: An implementation of randomized algorithms for computing the singular value, interpolative, and CUR decompositions of matrices on multi-core and GPU architectures
Simulations of large scale dynamical systems in multi-query or real-time contexts require efficient surrogate modelling techniques, as e.g. achieved via Model Order Reduction (MOR). Recently, symplectic methods like the complex singular value decomposition (cSVD) or the SVD-like decomposition have been developed for preserving Hamiltonian structure during MOR. In the current contribution, we show how symplectic structure preserving basis generation can be made more efficient with randomized matrix factorizations. We present a randomized complex SVD (rcSVD) algorithm and a randomized SVD-like (…
Solving a singular linear system for an individual vector solution is an ill-posed problem with a condition number infinity. From an alternative perspective, however, the general solution of a singular system is of a bounded sensitivity as a unique element in an affine Grassmannian. If a singular linear system is given through empirical data that are sufficiently accurate with a tight error bound, a properly formulated general numerical solution uniquely exists in the same affine Grassmannian, enjoys Lipschitz continuity and approximates the underlying exact solution with an accuracy in the sa…
Randomized block Krylov subspace methods form a powerful class of algorithms for computing the extreme eigenvalues of a symmetric matrix or the extreme singular values of a general matrix. The purpose of this paper is to develop new theoretical bounds on the performance of randomized block Krylov subspace methods for these problems. For matrices with polynomial spectral decay, the randomized block Krylov method can obtain an accurate spectral norm estimate using only a constant number of steps (that depends on the decay rate and the accuracy). Furthermore, the analysis reveals that the behavio…
In this paper, we introduce a randomized QLP decomposition called Rand-QLP. Operating on a matrix $\bf A$, Rand-QLP gives ${\bf A}={\bf QLP}^T$, where $\bf Q$ and $\bf P$ are orthonormal, and $\bf L$ is lower-triangular. Under the assumption that the rank of the input matrix is $k$, we derive several error bounds for Rand-QLP: bounds for the first $k$ approximate singular values and for the trailing block of the middle factor $\bf L$, which show that the decomposition is rank-revealing; bounds for the distance between approximate subspaces and the exact ones for all four fundamental subspaces …
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；University of Chinese Academy of Sciences（中国科学院大学）
We compared the regular Singular Value Decomposition (SVD), truncated SVD, Krylov method and Randomized PCA, in terms of time and space complexity. It is well-known that Krylov method and Randomized PCA only performs well when k << n, i.e. the number of eigenpair needed is far less than that of matrix size. We compared them for calculating all the eigenpairs. We also discussed the relationship between Principal Component Analysis and SVD.
Precision-induced Adaptive Randomized Low-Rank Approximation for SVD and Matrix Inversion
Nanjing University of Information Science and Technology（南京信息工程大学）；Xi’an Jiaotong University（西安交通大学）；Xiamen University（厦门大学）；Pengcheng Laboratory（鹏城实验室）
Eigensystem Realization Algorithm (ERA) is a data-driven approach for subspace system identification and is widely used in many areas of engineering. However, the computational cost of the ERA is dominated by a step that involves the singular value decomposition (SVD) of a large, dense matrix with block Hankel structure. This paper develops computationally efficient algorithms for reducing the computational cost of the SVD step by using randomized subspace iteration and exploiting the block Hankel structure of the matrix. We provide a detailed analysis of the error in the identified system mat…
