LSQR, a Lanczos bidiagonalization based Krylov subspace iterative method, and its mathematically equivalent CGLS applied to normal equations system, are commonly used for large-scale discrete ill-posed problems. It is well known that LSQR and CGLS have regularizing effects, where the number of iterations plays the role of the regularization parameter. However, it has long been unknown whether the regularizing effects are good enough to find best possible regularized solutions. Here a best possible regularized solution means that it is at least as accurate as the best regularized solution obtained by the truncated singular value decomposition (TSVD) method. In this paper, we establish bounds for the distance between the $k$-dimensional Krylov subspace and the $k$-dimensional dominant right singular space. They show that the Krylov subspace captures the dominant right singular space better for severely and moderately ill-posed problems than for mildly ill-posed problems. Our general conclusions are that LSQR has better regularizing effects for the first two kinds of problems than for the third kind, and a hybrid LSQR with additional regularization is generally needed for mildly ill-posed problems. Exploiting the established bounds, we derive an estimate for the accuracy of the rank $k$ approximation generated by Lanczos bidiagonalization. Numerical experiments illustrate that the regularizing effects of LSQR are good enough to compute best possible regularized solutions for severely and moderately ill-posed problems, stronger than our theory predicts, but they are not for mildly ill-posed problems and additional regularization is needed.
In the present work, we propose new tensor Krylov subspace method for ill posed linear tensor problems such as in color or video image restoration. Those methods are based on the tensor-tensor discrete cosine transform that gives fast tensor-tensor product computations. In particular, we will focus on the tensor discrete cosine versions of GMRES, Golub-Kahan bidiagonalisation and LSQR methods. The presented numerical tests show that the methods are very fast and give good accuracies when solving some linear tensor ill-posed problems.
For the large-scale linear discrete ill-posed problem $\min\|Ax-b\|$ or $Ax=b$ with $b$ contaminated by white noise, the Golub-Kahan bidiagonalization based LSQR method and its mathematically equivalent CGLS, the Conjugate Gradient (CG) method applied to $A^TAx=A^Tb$, are most commonly used. They have intrinsic regularizing effects, where the iteration number $k$ plays the role of regularization parameter. The long-standing fundamental question is: {\em Can LSQR and CGLS find 2-norm filtering best possible regularized solutions}? The author has given definitive answers to this question for sev…
Regularization Properties of the Krylov Iterative Solvers CGME and LSMR For Linear Discrete Ill-Posed Problems with an Application to Truncated Randomized SVDs
Modified Truncated Randomized Singular Value Decomposition (MTRSVD) Algorithms for Large Scale Discrete Ill-posed Problems with General-Form Regularization
We develop a generalized hybrid iterative approach for computing solutions to large-scale Bayesian inverse problems. We consider a hybrid algorithm based on the generalized Golub-Kahan bidiagonalization for computing Tikhonov regularized solutions to problems where explicit computation of the square root and inverse of the covariance kernel for the prior covariance matrix is not feasible. This is useful for large-scale problems where covariance kernels are defined on irregular grids or are only available via matrix-vector multiplication, e.g., those from the Matérn class. We show that iterates…
In this paper, we study a fractional-order variant of the asymptotical regularization method, called {\it Fractional Asymptotical Regularization (FAR)}, for solving linear ill-posed operator equations in a Hilbert space setting. We assign the method to the general linear regularization schema and prove that under certain smoothness assumptions, FAR with fractional order in the range $(1,2)$ yields an acceleration with respect to comparable order optimal regularization methods. Based on the one-step Adams-Moulton method, a novel iterative regularization scheme is developed for the numerical rea…
We consider a regularization concept for the solution of ill--posed operator equations, where the operator is composed of a continuous and a discontinuous operator. A particular application is level set regularization, where we develop a novel concept of minimizers. The proposed level set regularization is capable of handling changing topologies. A functional analytic framework explaining the splitting of topologies is given. The asymptotic limit of the level set regularization method is an evolution process, which is implemented numerically and the quality of the proposed algorithm is demonst…
In this paper, we consider the $α\| \cdot\|_{\ell_1}-β\| \cdot\|_{\ell_2}$ sparsity regularization with parameter $α\geqβ\geq0$ for nonlinear ill-posed inverse problems. We investigate the well-posedness of the regularization. Compared to the case where $α>β\geq0$, the results for the case $α=β\geq0$ are weaker due to the lack of coercivity and Radon-Riesz property of the regularization term. Under certain condition on the nonlinearity of $F$, we prove that every minimizer of $ α\| \cdot\|_{\ell_1}-β\| \cdot\|_{\ell_2}$ regularization is sparse. For the case $α>β\geq0$, if the exact solution i…
In this article we develop and analyze novel iterative regularization techniques for the solution of systems of nonlinear ill--posed operator equations. The basic idea consists in considering separately each equation of this system and incorporating a loping strategy. The first technique is a Kaczmarz-type method, equipped with a novel stopping criteria. The second method is obtained using an embedding strategy, and again a Kaczmarz-type approach. We prove well-posedness, stability and convergence of both methods.
Recently, inverse problems have attracted more and more attention in computational mathematics and become increasingly important in engineering applications. After the discretization, many of inverse problems are reduced to linear systems. Due to the typical ill-posedness of inverse problems, the reduced linear systems are often ill-posed, especially when their scales are large. This brings great computational difficulty. Particularly, a small perturbation in the right side of an ill-posed linear system may cause a dramatical change in the solution. Therefore, regularization methods should be …
In part I we introduced modified Landweber-Kaczmarz methods and have established a convergence analysis. In the present work we investigate three applications: an inverse problem related to thermoacoustic tomography, a nonlinear inverse problem for semiconductor equations, and a nonlinear problem in Schlieren tomography. Each application is considered in the framework established in the previous part. The novel algorithms show robustness, stability, computational efficiency and high accuracy.
This paper describes and compares some structure preserving techniques for the solution of linear discrete ill-posed problems with the t-product. A new randomized tensor singular value decomposition (R-tSVD) with a t-product is presented for low tubal rank tensor approximations. Regularization of linear inverse problems by truncated tensor eigenvalue decomposition (T-tEVD), truncated tSVD (T-tSVD), randomized T-tSVD (RT-tSVD), t-product Golub-Kahan bidiagonalization (tGKB) process, and t-product Lanczos (t-Lanczos) process are considered. A solution method that is based on reusing tensor Krylo…
Error Estimates for Arnoldo-Tikhonov Regularization for Ill-Posed Operator Equations
Institute for Industrial Mathematics, Kepler University（工业数学研究所，开普勒大学）；Johann Radon Institute for Computational and Applied Mathematics, Austrian Academy of Sciences（约翰·拉德纳计算与应用数学研究所，奥地利科学院）；Department of Mathematical Sciences, Kent State University（凯特州立大学数学科学系）
In this article we investigate the connection between regularization theory for inverse problems and dynamic programming theory. This is done by developing two new regularization methods, based on dynamic programming techniques. The aim of these methods is to obtain stable approximations to the solution of linear inverse ill-posed problems. We follow two different approaches and derive a continuous and a discrete regularization method. Regularization properties for both methods are proved as well as rates of convergence. A numerical benchmark problem concerning integral operators with convolut…
Shallow neural network yields regularization for ill-posed inverse problems
School of Mathematics and Statistics, Beijing Institute of Technology（北京理工大学数学与统计学院）；MSU-BIT-SMBU Joint Research Center of Applied Mathematics, Shenzhen MSU-BIT University（深圳北理莫斯科大学应用数学联合研究中心）
Two-parameter regularization of ill-posed spherical pseudo-differential equations in the space of continuous functions
Guangdong Province Key Laboratory of Computational Science, Sun Yat-sen University（中山大学）；Johann Radon Institute for Computational and Applied Mathematics, Austrian Academy of Sciences（奥地利科学院约翰·拉顿计算与应用数学研究所）；School of Mathematics and Statistics, University of New South Wales（新南威尔士大学数学与统计学院）
Regularization matrices for discrete ill-posed problems in several space-dimensions
University School（大学学校）；Geomathematics Key Laboratory of Sichuan, College of Management Science, Chengdu University of Technology（四川地球数学重点实验室，成都理工大学管理学院）；Department of Mathematics, SAPIENZA Università di Roma（罗马智慧大学数学系）；Department of Mathematical Sciences, Kent State University（肯特州立大学数理科学系）
