Eigensystem Realization Algorithm (ERA) is a data-driven approach for subspace system identification and is widely used in many areas of engineering. However, the computational cost of the ERA is dominated by a step that involves the singular value decomposition (SVD) of a large, dense matrix with block Hankel structure. This paper develops computationally efficient algorithms for reducing the computational cost of the SVD step by using randomized subspace iteration and exploiting the block Hankel structure of the matrix. We provide a detailed analysis of the error in the identified system matrices and the computational cost of the proposed algorithms. We demonstrate the accuracy and computational benefits of our algorithms on two test problems: the first involves a partial differential equation that models the cooling of steel rails, and the second is an application from power systems engineering.
We propose a new algorithm for the computation of a singular value decomposition (SVD) low-rank approximation of a matrix in the Matrix Product Operator (MPO) format, also called the Tensor Train Matrix format. Our tensor network randomized SVD (TNrSVD) algorithm is an MPO implementation of the randomized SVD algorithm that is able to compute dominant singular values and their corresponding singular vectors. In contrast to the state-of-the-art tensor-based alternating least squares SVD (ALS-SVD) and modified alternating least squares SVD (MALS-SVD) matrix approximation methods, TNrSVD can be u…
Randomized block Krylov method for approximation of truncated tensor SVD
Research Center of the Artificial Intelligence Institute, and Lab of Machine Learning and Knowledge Representation, Innopolis University（因诺波利斯大学人工智能研究所研究中心及机器学习和知识表示实验室）；Department of Teleinformatics Engineering, Federal University of Ceara（福塔莱萨塞阿拉联邦大学电信信息工程系）
EPF Lausanne（洛桑联邦理工学院）；Courant Institute of Mathematical Sciences, New York University（纽约大学柯朗数学科学研究所）；Center for Computational Mathematics, Flatiron Institute（Flatiron研究所计算数学中心）；Institute of Mathematics & Centre for Advanced Analytics and Predictive Sciences, University of Augsburg（奥格斯堡大学数学研究所与高级分析与预测科学中心）
The oriented singular value decomposition (O-SVD) proposed by Zeng and Ng provides a hybrid approach to the t-product based third-order tensor singular value decomposition with the transform matrix being a factor matrix of the higher order singular value decomposition. Continuing along this vein, this paper explores realizing the O-SVD more efficiently by the tensor-train rank-1 decomposition and gives a truncated O-SVD. Motivated by the success of probabilistic algorithms, we develop a randomized version of the O-SVD and present its detailed error analysis. The new algorithm has advantages in…
Simulations of large scale dynamical systems in multi-query or real-time contexts require efficient surrogate modelling techniques, as e.g. achieved via Model Order Reduction (MOR). Recently, symplectic methods like the complex singular value decomposition (cSVD) or the SVD-like decomposition have been developed for preserving Hamiltonian structure during MOR. In the current contribution, we show how symplectic structure preserving basis generation can be made more efficient with randomized matrix factorizations. We present a randomized complex SVD (rcSVD) algorithm and a randomized SVD-like (…
Big data analysis has become a crucial part of new emerging technologies such as the internet of things, cyber-physical analysis, deep learning, anomaly detection, etc. Among many other techniques, dimensionality reduction plays a key role in such analyses and facilitates feature selection and feature extraction. Randomized algorithms are efficient tools for handling big data tensors. They accelerate decomposing large-scale data tensors by reducing the computational complexity of deterministic algorithms and the communication among different levels of the memory hierarchy, which is the main bo…
Precision-induced Adaptive Randomized Low-Rank Approximation for SVD and Matrix Inversion
Nanjing University of Information Science and Technology（南京信息工程大学）；Xi’an Jiaotong University（西安交通大学）；Xiamen University（厦门大学）；Pengcheng Laboratory（鹏城实验室）
The tensor Singular Value Decomposition (t-SVD) for third order tensors that was proposed by Kilmer and Martin~\cite{2011kilmer} has been applied successfully in many fields, such as computed tomography, facial recognition, and video completion. In this paper, we propose a method that extends a well-known randomized matrix method to the t-SVD. This method can produce a factorization with similar properties to the t-SVD, but is more computationally efficient on very large datasets. We present details of the algorithm, theoretical results, and provide numerical results that show the promise of o…
KTH Royal Institute of Technology（瑞典皇家理工学院）；Academia Sinica（中央研究院）
Computing the top eigenvectors of a matrix is a problem of fundamental interest to various fields. While the majority of the literature has focused on analyzing the reconstruction error of low-rank matrices associated with the retrieved eigenvectors, in many applications one is interested in finding one vector with high Rayleigh quotient. In this paper we study the problem of approximating the top-eigenvector. Given a symmetric matrix $\mathbf{A}$ with largest eigenvalue $λ_1$, our goal is to find a vector \hu that approximates the leading eigenvector $\mathbf{u}_1$ with high accuracy, as meas…
Learning a dynamical system from input/output data is a fundamental task in the control design pipeline. In the partially observed setting there are two components to identification: parameter estimation to learn the Markov parameters, and system realization to obtain a state space model. In both sub-problems it is implicitly assumed that standard numerical algorithms such as the singular value decomposition (SVD) can be easily and reliably computed. When trying to fit a high-dimensional model to data, for example in the cyber-physical system setting, even computing an SVD is intractable. In t…
University of Oxford（牛津大学）；Cornell University（康奈尔大学）
The randomized singular value decomposition (SVD) is a popular and effective algorithm for computing a near-best rank $k$ approximation of a matrix $A$ using matrix-vector products with standard Gaussian vectors. Here, we generalize the randomized SVD to multivariate Gaussian vectors, allowing one to incorporate prior knowledge of $A$ into the algorithm. This enables us to explore the continuous analogue of the randomized SVD for Hilbert--Schmidt (HS) operators using operator-function products with functions drawn from a Gaussian process (GP). We then construct a new covariance kernel for GPs,…
IterativeCUR: Large Rank-Adaptive Approximation From a Small Recycled Sketch
Mathematical Institute at the University of Oxford（牛津大学数学研究所）；Institute of Mathematics at EPFL（洛桑联邦理工学院数学研究所）；Department of Mathematics and the Oden Institute at the University of Texas at Austin（德克萨斯大学奥斯汀分校数学系与奥登学院）
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；University of Chinese Academy of Sciences（中国科学院大学）
We compared the regular Singular Value Decomposition (SVD), truncated SVD, Krylov method and Randomized PCA, in terms of time and space complexity. It is well-known that Krylov method and Randomized PCA only performs well when k << n, i.e. the number of eigenpair needed is far less than that of matrix size. We compared them for calculating all the eigenpairs. We also discussed the relationship between Principal Component Analysis and SVD.
Low-rank matrix approximation is extremely useful in the analysis of data that arises in scientific computing, engineering applications, and data science. However, as data sizes grow, traditional low-rank matrix approximation methods, such as SVD and CPQR, are either prohibitively expensive or cannot provide sufficiently accurate results. A solution is to use randomized low-rank matrix approximation methods such as randomized SVD , and randomized LU decomposition on extremely large data sets. In this paper, we focus on the randomized LU decomposition method. First, we employ a reorthogonalizat…
The University of Texas at Austin（德克萨斯大学奥斯汀分校）；School of Information（信息学院）；Oden Institute for Computational Engineering and Sciences（奥登计算工程与科学研究所）
In a Big Data setting computing the dominant SVD factors is restrictive due to the main memory requirements. Recently introduced streaming Randomized SVD schemes work under the restrictive assumption that the singular value spectrum of the data has exponential decay. This is seldom true for any practical data. Although these methods are claimed to be applicable to scientific computations due to associated tail-energy error bounds, the approximation errors in the singular vectors and values are high when the aforementioned assumption does not hold. Furthermore from a practical perspective, over…
In recent years, the application of tensors has become more widespread in fields that involve data analytics and numerical computation. Due to the explosive growth of data, low-rank tensor decompositions have become a powerful tool to harness the notorious curse of dimensionality. The main forms of tensor decomposition include CP decomposition, Tucker decomposition, tensor train (TT) decomposition, etc. Each of the existing TT decomposition algorithms, including the TT-SVD and randomized TT-SVD, is successful in the field, but neither can both accurately and efficiently decompose large-scale s…
Fast randomized Kronecker tensor decomposition: algorithms and error analysis
Research Center of the Artificial Intelligence Institute, Innopolis University（因诺波利斯大学人工智能研究院研究中心）；Lab of Machine Learning and Knowledge Representation, Innopolis University（因诺波利斯大学机器学习与知识表示实验室）；Peoples’ Friendship University of Russia（俄罗斯人民友谊大学）；Department of Teleinformatics Engineering, Federal University of Ceara（塞阿拉联邦大学电信信息工程系）；School of Information and Communication Engineering, University of Electronic Science and Technology of China (UESTC)（电子科技大学信息与通信工程学院）
The QLP decomposition is one of the effective algorithms to approximate singular value decomposition (SVD) in numerical linear algebra. In this paper, we propose some single-pass randomized QLP decomposition algorithms for computing the low-rank matrix approximation. Compared with the deterministic QLP decomposition, the complexity of the proposed algorithms does not increase significantly and the system matrix needs to be accessed only once. Therefore, our algorithms are very suitable for a large matrix stored outside of memory or generated by stream data. In the error analysis, we give the b…
In this paper, we introduce a randomized QLP decomposition called Rand-QLP. Operating on a matrix $\bf A$, Rand-QLP gives ${\bf A}={\bf QLP}^T$, where $\bf Q$ and $\bf P$ are orthonormal, and $\bf L$ is lower-triangular. Under the assumption that the rank of the input matrix is $k$, we derive several error bounds for Rand-QLP: bounds for the first $k$ approximate singular values and for the trailing block of the middle factor $\bf L$, which show that the decomposition is rank-revealing; bounds for the distance between approximate subspaces and the exact ones for all four fundamental subspaces …
RSVDPACK: An implementation of randomized algorithms for computing the singular value, interpolative, and CUR decompositions of matrices on multi-core and GPU architectures
Modified Truncated Randomized Singular Value Decomposition (MTRSVD) Algorithms for Large Scale Discrete Ill-posed Problems with General-Form Regularization
University of Zagreb Faculty of Chemical Engineering and Technology（萨格勒布大学化学工程与技术学院）；University of Split Faculty of Electrical Engineering, Mechanical Engineering and Naval Architecture（斯普利特大学电气、机械与船舶工程学院）；University of Zagreb Faculty of Science（萨格勒布大学理学院）
Algorithm librla: A library of randomized linear algebra routines
Department of Applied Mathematics, University of Colorado Boulder（科罗拉多大学博尔德分校应用数学系）；National Institute of Standards and Technology (NIST)（美国国家标准与技术研究院）
