This paper is concerned with the solution of large-scale linear discrete ill-posed problems with error-contaminated data. Tikhonov regularization is a popular approach to determine meaningful approximate solutions of such problems. The choice of regularization matrix in Tikhonov regularization may significantly affect the quality of the computed approximate solution. This matrix should be chosen to promote the recovery of known important features of the desired solution, such as smoothness and monotonicity. We describe a novel approach to determine regularization matrices with desired properties by solving a matrix nearness problem. The constructed regularization matrix is the closest matrix in the Frobenius norm with a prescribed null space to a given matrix. Numerical examples illustrate the performance of the regularization matrices so obtained.
Regularization matrices for discrete ill-posed problems in several space-dimensions
University School（大学学校）；Geomathematics Key Laboratory of Sichuan, College of Management Science, Chengdu University of Technology（四川地球数学重点实验室，成都理工大学管理学院）；Department of Mathematics, SAPIENZA Università di Roma（罗马智慧大学数学系）；Department of Mathematical Sciences, Kent State University（肯特州立大学数理科学系）
In these lectures notes, we review our recent works addressing various problems of finding the nearest stable system to an unstable one. After the introduction, we provide some preliminary background, namely, defining Port-Hamiltonian systems and dissipative Hamiltonian systems and their properties, briefly discussing matrix factorizations, and describing the optimization methods that we will use in these notes. In the third chapter, we present our approach to tackle the distance to stability for standard continuous linear time invariant (LTI) systems. The main idea is to rely on the character…
A new approach to solving eigenvalue optimization problems for large structured matrices is proposed and studied. The class of optimization problems considered is related to computing structured pseudospectra and their extremal points, and to structured matrix nearness problems such as computing the structured distance to instability or to singularity. The structure can be a general linear structure and includes, for example, large matrices with a given sparsity pattern, matrices with given range and co-range, and Hamiltonian matrices. Remarkably, the eigenvalue optimization can be performed o…
Large kernel systems are prone to be ill-conditioned. Pivoted Cholesky decomposition (PCD) render a stable and efficient solution to the systems without a perturbation of regularization. This paper proposes a new PCD algorithm by tuning Cross Approximation (CA) algorithm to kernel matrices which merges the merits of PCD and CA, and proves as well as numerically exemplifies that it solves large kernel systems two-order more efficiently than those resorts to regularization. As a by-product, a diagonal-pivoted CA technique is also shown efficient in eigen-decomposition of large covariance matrice…
Many problems in Science and Engineering give rise to linear integral equations of the first kind with a smooth kernel. Discretization of the integral operator yields a matrix, whose singular values cluster at the origin. We describe the approximation of such matrices by adaptive cross approximation, which avoids forming the entire matrix. The choice of the number of steps of adaptive cross approximation is discussed. The discretized right-hand side represents data that commonly are contaminated by measurement error. Solution of the linear system of equations so obtained is not meaningful beca…
We consider the problem of recovering an unknown effectively $(s_1,s_2)$-sparse low-rank-$R$ matrix $X$ with possibly non-orthogonal rank-$1$ decomposition from incomplete and inaccurate linear measurements of the form $y = \mathcal A (X) + η$, where $η$ is an ineliminable noise. We first derive an optimization formulation for matrix recovery under the considered model and propose a novel algorithm, called Alternating Tikhonov regularization and Lasso (A-T-LA$\text{S}_{2,1}$), to solve it. The algorithm is based on a multi-penalty regularization, which is able to leverage both structures (low-…
In this work, we investigate various approaches that use learning from training data to solve inverse problems, following a bi-level learning approach. We consider a general framework for optimal inversion design, where training data can be used to learn optimal regularization parameters, data fidelity terms, and regularizers, thereby resulting in superior variational regularization methods. In particular, we describe methods to learn optimal $p$ and $q$ norms for ${\rm L}^p-{\rm L}^q$ regularization and methods to learn optimal parameters for regularization matrices defined by covariance kern…
The image deblurring problem consists of reconstructing images from blur and noise contaminated available data. In this AMS Notices article, we provide an overview of some well known numerical linear algebra techniques that are use for solving this problem. In particular, we start by carefully describing how to represent images, the process of blurring an image and modeling different kind of added noise. Then, we present regularization methods such as Tikhonov (on the standard and general form), Total Variation and other variations with sparse and edge preserving properties. Additionally, we b…
Standard regularization methods that are used to compute solutions to ill-posed inverse problems require knowledge of the forward model. In many real-life applications, the forward model is not known, but training data is readily available. In this paper, we develop a new framework that uses training data, as a substitute for knowledge of the forward model, to compute an optimal low-rank regularized inverse matrix directly, allowing for very fast computation of a regularized solution. We consider a statistical framework based on Bayes and empirical Bayes risk minimization to analyze theoretica…
A matrix $A\in\mathbb{C}^{n\times n}$ is diagonalizable if it has a basis of linearly independent eigenvectors. Since the set of nondiagonalizable matrices has measure zero, every $A\in \mathbb{C}^{n\times n}$ is the limit of diagonalizable matrices. We prove a quantitative version of this fact conjectured by E.B. Davies: for each $δ\in (0,1)$, every matrix $A\in \mathbb{C}^{n\times n}$ is at least $δ\|A\|$-close to one whose eigenvectors have condition number at worst $c_n/δ$, for some constants $c_n$ dependent only on $n$. Our proof uses tools from random matrix theory to show that the pseud…
