This paper describes and compares some structure preserving techniques for the solution of linear discrete ill-posed problems with the t-product. A new randomized tensor singular value decomposition (R-tSVD) with a t-product is presented for low tubal rank tensor approximations. Regularization of linear inverse problems by truncated tensor eigenvalue decomposition (T-tEVD), truncated tSVD (T-tSVD), randomized T-tSVD (RT-tSVD), t-product Golub-Kahan bidiagonalization (tGKB) process, and t-product Lanczos (t-Lanczos) process are considered. A solution method that is based on reusing tensor Krylov subspaces generated by the tGKB process is described. The regularization parameter is the number of iterations required by each method. The discrepancy principle is used to determine this parameter. Solution methods that are based on truncated iterations are compared with solution methods that combine Tikhonov regularization with the tGKB and t-Lanczos processes. Computed examples illustrate the performance of these methods when applied to image and gray-scale video restorations. Our new RT-tSVD method is seen to require less CPU time and yields restorations of higher quality than the T-tSVD method.
We propose a novel rank-adaptive higher-order orthogonal iteration (HOOI) algorithm to compute the truncated Tucker decomposition of higher-order tensors with a given error tolerance, and prove that the method is locally optimal and monotonically convergent. A series of numerical experiments related to both synthetic and real-world tensors are carried out to show that the proposed rank-adaptive HOOI algorithm is advantageous in terms of both accuracy and efficiency. Some further analysis on the HOOI algorithm and the classical alternating least squares method are presented to further understan…
A wide range of problems in computational science and engineering require estimation of sparse eigenvectors for high dimensional systems. Here, we propose two variants of the Truncated Orthogonal Iteration to compute multiple leading eigenvectors with sparsity constraints simultaneously. We establish numerical convergence results for the proposed algorithms using a perturbation framework, and extend our analysis to other existing alternatives for sparse eigenvector estimation. We then apply our algorithms to solve the sparse principle component analysis problem for a wide range of test dataset…
Modified Truncated Randomized Singular Value Decomposition (MTRSVD) Algorithms for Large Scale Discrete Ill-posed Problems with General-Form Regularization
Total least squares (TLS) methods have been widely used in data fitting. Compared with the least squares method, for TLS problem we takes into account not only the observation errors, but also the errors in the measurement matrix. This is more realistic in practical applications. For the large-scale discrete ill-posed problem $Ax \approx b$, we introduce the quantum-inspired techniques to approximate the truncated total least squares (TTLS) solution. We analyze the accuracy of the quantum-inspired truncated total least squares algorithm and perform numerical experiments to demonstrate the effi…
Regularization Properties of the Krylov Iterative Solvers CGME and LSMR For Linear Discrete Ill-Posed Problems with an Application to Truncated Randomized SVDs
In this paper, we consider a modified Levenberg--Marquardt method for solving an ill-posed inverse problem where the forward mapping is not Gâteaux differentiable. By relaxing the standard assumptions for the classical smooth setting, we derive asymptotic stability estimates that are then used to prove the convergence of the proposed method. This method can be applied to an inverse source problem for a non-smooth semilinear elliptic PDE where a Bouligand subdifferential can be used in place of the non-existing Fréchet derivative, and we show that the corresponding Bouligand-Levenberg-Marquardt…
The joint bidiagonalization process of a matrix pair $\{A,L\}$ can be used to develop iterative regularization algorithms for large scale ill-posed problems in general-form Tikhonov regularization $\min_x\left\{\|Ax-b\|_{2}^2+λ^2\|Lx\|_{2}^2\right\}$ or the essentially equivalent one $\min\|Lx\|_{2} \ \mbox{\rm s.t.} \ x\in\mathcal{S} = \{x|\ \|Ax-b\|_{2}\leq η\|e\|_{2}\}$, where $e$ is a Gaussian white noise, $L$ is a regularization matrix and $η>1$ slightly. A bottleneck of the algorithms is that a large scale inner least squares problem with $(A^{T}, L^{T})^{T}$ as the coefficient matrix mu…
In this article we combine the projective Landweber method, recently proposed by the authors, with Kaczmarz's method for solving systems of non-linear ill-posed equations. The underlying assumption used in this work is the tangential cone condition. We show that the proposed iteration is a convergent regularization method. Numerical tests are presented for a non-linear inverse problem related to the Dirichlet-to-Neumann map, indicating a superior performance of the proposed method when compared with other well established iterations. Our preliminary investigation indicates that the resulting i…
The truncated singular value decomposition may be used to find the solution of linear discrete ill-posed problems in conjunction with Tikhonov regularization and requires the estimation of a regularization parameter that balances between the sizes of the fit to data function and the regularization term. The unbiased predictive risk estimator is one suggested method for finding the regularization parameter when the noise in the measurements is normally distributed with known variance. In this paper we provide an algorithm using the unbiased predictive risk estimator that automatically finds bot…
We propose a stochastic gradient descent approach with partitioned-truncated singular value decomposition for large-scale inverse problems of magnetic modulus data. Motivated by a uniqueness theorem in gravity inverse problem and realizing the similarity between gravity and magnetic inverse problems, we propose to solve the level-set function modeling the volume susceptibility distribution from the nonlinear magnetic modulus data. To deal with large-scale data, we employ a mini-batch stochastic gradient descent approach with random reshuffling when solving the optimization problem of the inver…
A commonly used technique for the higher-order PageRank problem is the power method that is computationally intractable for large-scale problems. The truncated power method proposed recently provides us with another idea to solve this problem, however, its accuracy and efficiency can be poor in practical computations. In this work, we revisit the higher-order PageRank problem and consider how to solve it efficiently. The contribution of this work is as follows. First, we accelerate the truncated power method for high-order PageRank. In the improved version, it is neither to form and store the …
In this paper, we present explicit expressions for the mixed and componentwise condition numbers of the truncated total least squares (TTLS) solution of $A\boldsymbol{x} \approx \boldsymbol{b} $ under the genericity condition, where $A$ is a $m\times n$ real data matrix and $\boldsymbol{b}$ is a real $m$-vector. Moreover, we reveal that normwise, componentwise and mixed condition numbers for the TTLS problem can recover the previous corresponding counterparts for the total least squares (TLS) problem when the truncated level of for the TTLS problem is $n$. When $A$ is a structured matrix, the …
We derive and analyse a new variant of the iteratively regularized Landweber iteration, for solving linear and nonlinear ill-posed inverse problems. The method takes into account training data, which are used to estimate the interior of a black box, which is used to define the iteration process. We prove convergence and stability for the scheme in infinite dimensional Hilbert spaces. These theoretical results are complemented by several numerical experiments for solving linear inverse problems for the Radon transform and a nonlinear inverse problem for Schlieren tomography.
In this paper, we propose the tensor Noda iteration (NI) and its inexact version for solving the eigenvalue problem of a particular class of tensor pairs called generalized $\mathcal{M}$-tensor pairs. A generalized $\mathcal{M}$-tensor pair consists of a weakly irreducible nonnegative tensor and a nonsingular $\mathcal{M}$-tensor within a linear combination. It is shown that any generalized $\mathcal{M}$-tensor pair admits a unique positive generalized eigenvalue with a positive eigenvector. A modified tensor Noda iteration(MTNI) is developed for extending the Noda iteration for nonnegative ma…
The nonlinear eigen-problem $ Ax+F(x)=λx$ is studied where $A$ is an $n\times n$ irreducible Stieltjes matrix. Under certain conditions, this problem has a unique positive solution. We show that, starting from a multiple of the positive eigenvector of $A$, the Newton-like iteration for this problem converges monotonically. Numerical results illustrate the effectiveness of this Newton-like method.
For a linear complementarity problem, we present a relaxaiton accelerated two-sweep matrix splitting iteration method. The convergence analysis illustrates that the proposed method converges to the exact solution of the linear complementarity problem when the system matrix is an $H_+$-matrix and the convergence conditions are given. Numerical experiments show that the proposed method is more efficient than the existing ones.
Stability estimates for the regularized inversion of the truncated Hilbert transform
Seminar for Applied Mathematics, ETH Zürich（苏黎世联邦理工学院应用数学研讨会）；Department of Nuclear Medicine, Vrije Universiteit Brussel（布鲁塞尔自由大学核医学系）；Department of Mathematics, University of Central Florida（中佛罗里达大学数学系）
This report shows on real data that the direct methods such as LDL decomposition and Gaussian elimination for solving linear systems with ill-conditioned matrices provide inaccurate results due to divisions by very small numbers, which in turn results in peaking phenomena and large estimation errors. Richardson iteration provides accurate results without peaking phenomena since division by small numbers is absent in the Richardson approach. In addition, two preconditioners are considered and compared in the Richardson iteration: 1) the simplest and robust preconditioner based on the maximum ro…
