This paper presents a randomized quaternion singular value decomposition (QSVD) algorithm for low-rank matrix approximation problems, which are widely used in color face recognition, video compression, and signal processing problems. With quaternion normal distribution based random sampling, the randomized QSVD algorithm projects a high-dimensional data to a low-dimensional subspace and then identifies an approximate range subspace of the quaternion matrix. The key statistical properties of quaternion Wishart distribution are proposed and used to perform the approximation error analysis of the algorithm. Theoretical results show that the randomized QSVD algorithm can trace dominant singular value decomposition triplets of a quaternion matrix with acceptable accuracy. Numerical experiments also indicate the rationality of proposed theories. Applied to color face recognition problems, the randomized QSVD algorithm obtains higher recognition accuracies and behaves more efficient than the known Lanczos-based partial QSVD and a quaternion version of fast frequent directions algorithm.
Pass-efficient Randomized Algorithms for Low-rank Approximation of Quaternion Matrices
Innopolis University（伊诺波利斯大学）；Research Center of the Artificial Intelligence Institute, and Lab of Machine Learning and Knowledge Representation, Innopolis University（伊诺波利斯大学人工智能研究所研究中心及机器学习与知识表示实验室）
A flexible transform-based tensor product named $\star_{\rm{QT}}$-product for $L$th-order ($L\geq 3$) quaternion tensors is proposed. Based on the $\star_{\rm{QT}}$-product, we define the corresponding singular value decomposition named TQt-SVD and the rank named TQt-rank of the $L$th-order ($L\geq 3$) quaternion tensor. Furthermore, with orthogonal quaternion transformations, the TQt-SVD can provide the best TQt-rank-$s$ approximation of any $L$th-order ($L\geq 3$) quaternion tensor. In the experiments, we have verified the effectiveness of the proposed TQt-SVD in the application of the best …
Higher-order singular value decomposition (HOSVD) is one of the most efficient tensor decomposition techniques. It has the salient ability to represent high_dimensional data and extract features. In more recent years, the quaternion has proven to be a very suitable tool for color pixel representation as it can well preserve cross-channel correlation of color channels. Motivated by the advantages of the HOSVD and the quaternion tool, in this paper, we generalize the HOSVD to the quaternion domain and define quaternion-based HOSVD (QHOSVD). Due to the non-commutability of quaternion multiplicati…
The generalized singular value decomposition (GSVD) is a valuable tool that has many applications in computational science. However, computing the GSVD for large-scale problems is challenging. Motivated by applications in hyper-differential sensitivity analysis (HDSA), we propose new randomized algorithms for computing the GSVD which use randomized subspace iteration and weighted QR factorization. Detailed error analysis is given which provides insight into the accuracy of the algorithms and the choice of the algorithmic parameters. We demonstrate the performance of our algorithms on test matr…
The oriented singular value decomposition (O-SVD) proposed by Zeng and Ng provides a hybrid approach to the t-product based third-order tensor singular value decomposition with the transform matrix being a factor matrix of the higher order singular value decomposition. Continuing along this vein, this paper explores realizing the O-SVD more efficiently by the tensor-train rank-1 decomposition and gives a truncated O-SVD. Motivated by the success of probabilistic algorithms, we develop a randomized version of the O-SVD and present its detailed error analysis. The new algorithm has advantages in…
In this work, we develop efficient solvers for linear inverse problems based on randomized singular value decomposition (RSVD). This is achieved by combining RSVD with classical regularization methods, e.g., truncated singular value decomposition, Tikhonov regularization, and general Tikhonov regularization with a smoothness penalty. One distinct feature of the proposed approach is that it explicitly preserves the structure of the regularized solution in the sense that it always lies in the range of a certain adjoint operator. We provide error estimates between the approximation and the exact …
The tensor Singular Value Decomposition (t-SVD) for third order tensors that was proposed by Kilmer and Martin~\cite{2011kilmer} has been applied successfully in many fields, such as computed tomography, facial recognition, and video completion. In this paper, we propose a method that extends a well-known randomized matrix method to the t-SVD. This method can produce a factorization with similar properties to the t-SVD, but is more computationally efficient on very large datasets. We present details of the algorithm, theoretical results, and provide numerical results that show the promise of o…
Quaternion Maximum-Volume Submatrix Selection with Applications to Multichannel Imaging and Visual Data
Institute of Data Science and Artificial Intelligence, Innopolis University（因诺波利斯大学数据科学与人工智能研究所）；Department of Mathematics, Faculty of Science, Hong Kong Baptist University, Hong Kong, China（香港浸会大学理学院数学系）
In this paper, we focus on developing randomized algorithms for the computation of low multilinear rank approximations of tensors based on the random projection and the singular value decomposition. Following the theory of the singular values of sub-Gaussian matrices, we make a probabilistic analysis for the error bounds for the randomized algorithm. We demonstrate the effectiveness of proposed algorithms via several numerical examples.
The randomized Gauss--Seidel method and its extension have attracted much attention recently and their convergence rates have been considered extensively. However, the convergence rates are usually determined by upper bounds, which cannot fully reflect the actual convergence. In this paper, we make a detailed analysis of their convergence behaviors. The analysis shows that the larger the singular value of $A$ is, the faster the error decays in the corresponding singular vector space, and the convergence directions are mainly driven by the large singular values at the beginning, then gradually …
In this paper, we analyze the convergence behavior of the randomized extended Kaczmarz (REK) method for all types of linear systems (consistent or inconsistent, overdetermined or underdetermined, full-rank or rank-deficient). The analysis shows that the larger the singular value of $A$ is, the faster the error decays in the corresponding right singular vector space, and as $k\rightarrow\infty$, $x_{k}-x_{\star}$ tends to the right singular vector corresponding to the smallest singular value of $A$, where $x_{k}$ is the $k$th approximation of the REK method and $x_{\star}$ is the minimum $\ell_…
Shanghai University（上海大学）；Research Institute of USV Engineering（无人艇工程研究院）；Jiangsu Normal University（江苏师范大学）；Research Institute of Mathematical Science（数学科学研究院）
Quaternion singular value decomposition (QSVD) is a robust technique of digital watermarking which can extract high quality watermarks from watermarked images with low distortion. In this paper, QSVD technique is further investigated and an efficient robust watermarking scheme is proposed. The improved algebraic structure-preserving method is proposed to handle the problem of "explosion of complexity" occurred in the conventional QSVD design. Secret information is transmitted blindly by incorporating in QSVD two new strategies, namely, coefficient pair selection and adaptive embedding. Unlike …
As a new color image representation tool, quaternion has achieved excellent results in the color image processing, because it treats the color image as a whole rather than as a separate color space component, thus it can make full use of the high correlation among RGB channels. Recently, low-rank quaternion matrix completion (LRQMC) methods have proven very useful for color image inpainting. In this paper, we propose three novel LRQMC methods based on three quaternion-based bilinear factor (QBF) matrix norm minimization models. Specifically, we define quaternion double Frobenius norm (Q-DFN), …
Modified Truncated Randomized Singular Value Decomposition (MTRSVD) Algorithms for Large Scale Discrete Ill-posed Problems with General-Form Regularization
Quaternion matrices are employed successfully in many color image processing applications. In particular, a pure quaternion matrix can be used to represent red, green and blue channels of color images. A low-rank approximation for a pure quaternion matrix can be obtained by using the quaternion singular value decomposition. However, this approximation is not optimal in the sense that the resulting low-rank approximation matrix may not be pure quaternion, i.e., the low-rank matrix contains real component which is not useful for the representation of a color image. The main contribution of this …
The QLP decomposition is one of the effective algorithms to approximate singular value decomposition (SVD) in numerical linear algebra. In this paper, we propose some single-pass randomized QLP decomposition algorithms for computing the low-rank matrix approximation. Compared with the deterministic QLP decomposition, the complexity of the proposed algorithms does not increase significantly and the system matrix needs to be accessed only once. Therefore, our algorithms are very suitable for a large matrix stored outside of memory or generated by stream data. In the error analysis, we give the b…
Randomized Kaczmarz is a simple iterative method for finding solutions of linear systems $Ax = b$. We point out that the arising sequence $(x_k)_{k=1}^{\infty}$ tends to converge to the solution $x$ in an interesting way: generically, as $k \rightarrow \infty$, $x_k - x$ tends to the singular vector of $A$ corresponding to the smallest singular value. This has interesting consequences: in particular, the error analysis of Strohmer \& Vershynin is optimal. It also quantifies the `pre-convergence' phenomenon where the method initially seems to converge faster. This fact also allows for a fast co…
We propose a new algorithm for the computation of a singular value decomposition (SVD) low-rank approximation of a matrix in the Matrix Product Operator (MPO) format, also called the Tensor Train Matrix format. Our tensor network randomized SVD (TNrSVD) algorithm is an MPO implementation of the randomized SVD algorithm that is able to compute dominant singular values and their corresponding singular vectors. In contrast to the state-of-the-art tensor-based alternating least squares SVD (ALS-SVD) and modified alternating least squares SVD (MALS-SVD) matrix approximation methods, TNrSVD can be u…
University of Oxford（牛津大学）；Cornell University（康奈尔大学）
The randomized singular value decomposition (SVD) is a popular and effective algorithm for computing a near-best rank $k$ approximation of a matrix $A$ using matrix-vector products with standard Gaussian vectors. Here, we generalize the randomized SVD to multivariate Gaussian vectors, allowing one to incorporate prior knowledge of $A$ into the algorithm. This enables us to explore the continuous analogue of the randomized SVD for Hilbert--Schmidt (HS) operators using operator-function products with functions drawn from a Gaussian process (GP). We then construct a new covariance kernel for GPs,…
The large-scale focusing inversion of gravity and magnetic potential field data using $L_1$-norm regularization is considered. The use of the randomized singular value decomposition methodology facilitates tackling the computational challenge that arises in the solution of these large-scale inverse problems. As such the powerful randomized singular value decomposition is used for the numerical solution of all linear systems required in the algorithm. A comprehensive comparison of the developed methodology for the inversion of magnetic and gravity data is presented. These results indicate that …
Regression analysis-based approaches have been widely studied for face recognition (FR) in the past several years. More recently, to better deal with some difficult conditions such as occlusions and illumination, nuclear norm based matrix regression methods have been proposed to characterize the low-rank structure of the error image, which generalize the one-dimensional, pixel-based error model to the two-dimensional structure. These methods, however, are inherently devised for grayscale image based FR and without exploiting the color information which is proved beneficial for FR of color face…
Simulations of large scale dynamical systems in multi-query or real-time contexts require efficient surrogate modelling techniques, as e.g. achieved via Model Order Reduction (MOR). Recently, symplectic methods like the complex singular value decomposition (cSVD) or the SVD-like decomposition have been developed for preserving Hamiltonian structure during MOR. In the current contribution, we show how symplectic structure preserving basis generation can be made more efficient with randomized matrix factorizations. We present a randomized complex SVD (rcSVD) algorithm and a randomized SVD-like (…
RSVDPACK: An implementation of randomized algorithms for computing the singular value, interpolative, and CUR decompositions of matrices on multi-core and GPU architectures
