In this paper we the formulation of inverse problems as constrained minimization problems and their iterative solution by gradient or Newton type. We carry out a convergence analysis in the sense of regularization methods and discuss applicability to the problem of identifying the spatially varying diffusivity in an elliptic PDE from different sets of observations. Among these is a novel hybrid imaging techology known as impedance acoustic tomography, for which we provide numerical experiments.
We consider the Adaptive Regularization with Cubics approach for solving nonconvex optimization problems and propose a new variant based on inexact Hessian information chosen dynamically. The theoretical analysis of the proposed procedure is given. The key property of ARC framework, constituted by optimal worst-case function/derivative evaluation bounds for first- and second-order critical point, is guaranteed. Application to large-scale finite-sum minimization based on subsampled Hessian is discussed and analyzed in both a deterministic and probabilistic manner and equipped with numerical exp…
This paper deals with regularized Newton methods, a flexible class of unconstrained optimization algorithms that is competitive with line search and trust region methods and potentially combines attractive elements of both. The particular focus is on combining regularization with limited memory quasi-Newton methods by exploiting the special structure of limited memory algorithms. Global convergence of regularization methods is shown under mild assumptions and the details of regularized limited memory quasi-Newton updates are discussed including their compact representations. Numerical results …
In this paper, we consider the $\ell_0$ minimization problem whose objective function is the sum of $\ell_0$-norm and convex differentiable function. A variable metric type method which combines the PIHT method and the skill in quasi-newton method, named variable metric extrapolation proximal iterative hard thresholding (VMEPIHT) method, is proposed. Then we analyze its convergence, linear convergence rate and superlinear convergence rate under appropriate assumptions. Finally, we conduct numerical experiments on compressive sensing problem and CT image reconstruction problem to confirm VMPIHT…
In this paper we extend a recent idea of formulating and regularizing inverse problems as minimization problems, so without using a forward operator, thus avoiding explicit evaluation of a parameter-to-state map. We do so by rephrasing three application examples in this minimization form, namely (a) electrical impedance tomography with the complete electrode model (b) identification of a nonlinear magnetic permeability from magnetic flux measurements (c) localization of sound sources from microphone array measurements. To establish convergence of the proposed regularization approach for these …
Western Norway University of Applied Sciences（西挪威应用科学大学）；University of Warsaw（华沙大学）；University of Bergen（卑尔根大学）
We propose a set of iterative regularization algorithms for the TV-Stokes model to restore images from noisy images with Gaussian noise. These are some extensions of the iterative regularization algorithm proposed for the classical Rudin-Osher-Fatemi (ROF) model for image reconstruction, a single step model involving a scalar field smoothing, to the TV-Stokes model for image reconstruction, a two steps model involving a vector field smoothing in the first and a scalar field smoothing in the second. The iterative regularization algorithms proposed here are Richardson's iteration like. We have e…
We propose and investigate efficient numerical methods for inverse problems related to Magnetic Resonance Imaging (MRI). Our goal is to extend the recent convergence results for the Landweber-Kaczmarz method obtained in [Haltmeier, Leitao, Scherzer 2007], in order to derive a convergent iterative regularization method for an inverse problem in MRI.
In this paper we investigate the problem of recovering the source term in an elliptic system from a measurement of the state on a part of the boundary. For the particular interest in reconstructing probably discontinuous sources, we use the standard least squares method with the total variation regularization. The finite element method is then applied to discretize the minimization problem, we show the stability and the convergence of this technique. Furthermore, we have proposed an algorithm to stably solve the minimization problem. We prove the iterate sequence generated by the derived algor…
Many inverse problems are concerned with the estimation of non-negative parameter functions. In this paper, in order to obtain non-negative stable approximate solutions to ill-posed linear operator equations in a Hilbert space setting, we develop two novel non-negativity preserving iterative regularization methods. They are based on fixed point iterations in combination with preconditioning ideas. In contrast to the projected Landweber iteration, for which only weak convergence can be shown for the regularized solution when the noise level tends to zero, the introduced regularization methods e…
The affine rank minimization (ARM) problem is well known for both its applications and the fact that it is NP-hard. One of the most successful approaches, yet arguably underrepresented, is iteratively reweighted least squares (IRLS), more specifically $\mathrm{IRLS}$-$0$. Despite comprehensive empirical evidence that it overall outperforms nuclear norm minimization and related methods, it is still not understood to a satisfying degree. In particular, the significance of a slow decrease of the therein appearing regularization parameter denoted $γ$ poses interesting questions. While commonly equ…
Signals and images with discontinuities appear in many problems in such diverse areas as biology, medicine, mechanics, and electrical engineering. The concrete data are often discrete, indirect and noisy measurements of some quantities describing the signal under consideration. A frequent task is to find the segments of the signal or image which corresponds to finding the discontinuities or jumps in the data. Methods based on minimizing the piecewise constant Mumford-Shah functional -- whose discretized version is known as Potts functional -- are advantageous in this scenario, in particular, i…
Connected with the rise of interest in inverse problems is the development and analysis of regularization methods, which are a necessity due to the ill-posedness of inverse problems. Tikhonov-type regularization methods are very popular in this regard. However, its direct implementation for large-scale linear or non-linear problems is a non-trivial task. In such scenarios, iterative regularization methods usually serve as a better alternative. In this paper we propose a new iterative regularization method which uses descent directions, different from the usual gradient direction, that enable a…
In this paper we propose a new class of iterative regularization methods for solving ill-posed linear operator equations. The prototype of these iterative regularization methods is in the form of second order evolution equation with a linear vanishing damping term, which can be viewed not only as an extension of the asymptotical regularization, but also as a continuous analog of the Nesterov's acceleration scheme. New iterative regularization methods are derived from this continuous model in combination with damped symplectic numerical schemes. The regularization property as well as convergenc…
Recently, inverse problems have attracted more and more attention in computational mathematics and become increasingly important in engineering applications. After the discretization, many of inverse problems are reduced to linear systems. Due to the typical ill-posedness of inverse problems, the reduced linear systems are often ill-posed, especially when their scales are large. This brings great computational difficulty. Particularly, a small perturbation in the right side of an ill-posed linear system may cause a dramatical change in the solution. Therefore, regularization methods should be …
Affine sum-of-ranks minimization (ASRM) generalizes the affine rank minimization (ARM) problem from matrices to tensors. Here, the interest lies in the ranks of a family $\mathcal{K}$ of different matricizations. Transferring our priorly discussed results on asymptotic log-det rank minimization, we show that iteratively reweighted least squares with weight strength $p = 0$ remains a, theoretically and practically, particularly viable method denoted as $\mathrm{IRLS}$-$0\mathcal{K}$. As in the matrix case, we prove global convergence of asymptotic minimizers of the log-det sum-of-ranks function…
A least-squares method for solving the hyperbolic Monge-Ampère equation with transport boundary condition is introduced. The method relies on an iterative procedure for the gradient of the solution, the so-called mapping. By formulating error functionals for the interior domain, the boundary, both separately and as linear combination, three minimization problems are solved iteratively to compute the mapping. After convergence, a fourth minimization problem, to compute the solution of the Monge-Ampère equation, is solved. The approach is based on a least-squares method for the elliptic Monge-Am…
In this paper, we introduce a class of nonsmooth nonconvex least square optimization problem using convex analysis tools and we propose to use the iterative minimization-majorization (MM) algorithm on a convex set with initializer away from the origin to find an optimal point for the optimization problem. For this, first we use an approach to construct a class of convex majorizers which approximate the value of non-convex cost function on a convex set. The convergence of the iterative algorithm is guaranteed when the initial point $x^{(0)}$ is away from the origin and the iterative points $x^{…
We investigate the Cauchy problem for linear elliptic operators with $C^\infty$-coefficients at a regular set $Ω\subset R^2$, which is a classical example of an ill-posed problem. The Cauchy data are given at the manifold $Γ\subset \partialΩ$ and our goal is to reconstruct the trace of the $H^1(Ω)$ solution of an elliptic equation at $\partial Ω/ Γ$. The method proposed here composes the segmenting Mann iteration with a fixed point equation associated with the elliptic Cauchy problem. Our algorithm generalizes the iterative method developed by Maz'ya et al., who proposed a method based on solv…
Low-rank matrix recovery via iteratively reweighted least squares minimization
Johann Radon Institute for Computational and Applied Mathematics, Austrian Academy of Sciences（奥地利科学院约翰·拉东计算与应用数学研究所）；Hausdorff Center for Mathematics & Institute for Numerical Simulation, University of Bonn（波恩大学霍夫多夫数学中心与数值模拟研究所）；Mathematics Department, Courant Institute of Mathematical Sciences, New York University（纽约大学库朗数学科学研究所数学系）
A generalized matrix Krylov subspace method for TV regularization
Faculté des Sciences et Techniques-Gueliz, Laboratoire de Mathématiques Appliquées et Informatique（盖利兹科学技术学院，应用数学与计算机实验室）；Université de Lille Nord de France, L.M.P.A, ULCO（法国北部里尔大学，L.M.P.A，ULCO）
These lecture notes for a graduate class present the regularization theory for linear and nonlinear ill-posed operator equations in Hilbert spaces. Covered are the general framework of regularization methods and their analysis via spectral filters as well as the concrete examples of Tikhonov regularization, Landweber iteration, regularization by discretization for linear inverse problems. In the nonlinear setting, Tikhonov regularization and iterative regularization (Landweber, Levenberg-Marquardt, and iteratively regularized Gauß-Newton methods) are discussed. The necessary background from fu…
Fast Linearized Alternating Direction Minimization Algorithm with Adaptive Parameter Selection for Multiplicative Noise Removal
Department of Mathematics, Third Military Medical University, Chongqing 400038, Chongqing, People's Republic of China（数学系，第三军医大学）；Department of Mathematics and System, School of Sciences, National University of Defense Technology, Changsha 410073, Hunan, People's Republic of China（数学与系统系，国防科技大学）
