We analyzed the performance of a biologically inspired algorithm called the Corrected Projections Algorithm (CPA) when a sparseness constraint is required to unambiguously reconstruct an observed signal using atoms from an overcomplete dictionary. By changing the geometry of the estimation problem, CPA gives an analytical expression for a binary variable that indicates the presence or absence of a dictionary atom using an L2 regularizer. The regularized solution can be implemented using an efficient real-time Kalman-filter type of algorithm. The smoother L2 regularization of CPA makes it very robust to noise, and CPA outperforms other methods in identifying known atoms in the presence of strong novel atoms in the signal.
University of Innsbruck（因斯布鲁克大学）；University of Idaho（爱达荷大学）
Inverse problems arise in a variety of imaging applications including computed tomography, non-destructive testing, and remote sensing. The characteristic features of inverse problems are the non-uniqueness and instability of their solutions. Therefore, any reasonable solution method requires the use of regularization tools that select specific solutions and at the same time stabilize the inversion process. Recently, data-driven methods using deep learning techniques and neural networks demonstrated to significantly outperform classical solution methods for inverse problems. In this chapter, w…
We analyze sparse frame based regularization of inverse problems by means of a diagonal frame decomposition (DFD) for the forward operator, which generalizes the SVD. The DFD allows to define a non-iterative (direct) operator-adapted frame thresholding approach which we show to provide a convergent regularization method with linear convergence rates. These results will be compared to the well-known analysis and synthesis variants of sparse $\ell^1$-regularization which are usually implemented thorough iterative schemes. If the frame is a basis (non-redundant case), the three versions of sparse…
Friedrich-Alexander Universität Erlangen-Nürnberg（埃尔朗根-纽伦堡大学）
This paper discusses basic results and recent developments on variational regularization methods, as developed for inverse problems. In a typical setup we review basic properties needed to obtain a convergent regularization scheme and further discuss the derivation of quantitative estimates respectively needed ingredients such as Bregman distances for convex functionals. In addition to the approach developed for inverse problems we will also discuss variational regularization in machine learning and work out some connections to the classical regularization theory. In particular we will discuss…
We propose a sparse reconstruction framework for solving inverse problems. Opposed to existing sparse regularization techniques that are based on frame representations, we train an encoder-decoder network by including an $\ell^1$-penalty. We demonstrate that the trained decoder network allows sparse signal reconstruction using thresholded encoded coefficients without losing much quality of the original image. Using the sparse synthesis prior, we propose minimizing the $\ell^1$-Tikhonov functional, which is the sum of a data fitting term and the $\ell^1$-norm of the synthesis coefficients, and …
We propose and investigate efficient numerical methods for inverse problems related to Magnetic Resonance Imaging (MRI). Our goal is to extend the recent convergence results for the Landweber-Kaczmarz method obtained in [Haltmeier, Leitao, Scherzer 2007], in order to derive a convergent iterative regularization method for an inverse problem in MRI.
These lecture notes for a graduate class present the regularization theory for linear and nonlinear ill-posed operator equations in Hilbert spaces. Covered are the general framework of regularization methods and their analysis via spectral filters as well as the concrete examples of Tikhonov regularization, Landweber iteration, regularization by discretization for linear inverse problems. In the nonlinear setting, Tikhonov regularization and iterative regularization (Landweber, Levenberg-Marquardt, and iteratively regularized Gauß-Newton methods) are discussed. The necessary background from fu…
In this paper we consider new regularization methods for linear inverse problems of dynamic type. These methods are based on dynamic programming techniques for linear quadratic optimal control problems. Two different approaches are followed: a continuous and a discrete one. We prove regularization properties and also obtain rates of convergence for the methods derived from both approaches. A numerical example concerning the dynamic EIT problem is used to illustrate the theoretical results.
Technical University of Denmark（丹麦技术大学）；Virginia Tech（弗吉尼亚理工大学）
In this work, we describe a new approach that uses deep neural networks (DNN) to obtain regularization parameters for solving inverse problems. We consider a supervised learning approach, where a network is trained to approximate the mapping from observation data to regularization parameters. Once the network is trained, regularization parameters for newly obtained data can be computed by efficient forward propagation of the DNN. We show that a wide variety of regularization functionals, forward models, and noise models may be considered. The network-obtained regularization parameters can be c…
Recently, a large number of efficient deep learning methods for solving inverse problems have been developed and show outstanding numerical performance. For these deep learning methods, however, a solid theoretical foundation in the form of reconstruction guarantees is missing. In contrast, for classical reconstruction methods, such as convex variational and frame-based regularization, theoretical convergence and convergence rate results are well established. In this paper, we introduce deep synthesis regularization (DESYRE) using neural networks as nonlinear synthesis operator bridging the ga…
Universität Heidelberg（海德堡大学）；Hochschule Darmstadt（达姆施塔特应用技术大学）；Helmholtz Zentrum München（慕尼黑亥姆霍兹中心）
In this paper, we consider the sparse regularization of manifold-valued data with respect to an interpolatory wavelet/multiscale transform. We propose and study variational models for this task and provide results on their well-posedness. We present algorithms for a numerical realization of these models in the manifold setup. Further, we provide experimental results to show the potential of the proposed schemes for applications.
In this paper, we consider the asymptotical regularization with convex constraints for nonlinear ill-posed problems. The method allows to use non-smooth penalty terms, including the L1-like and the total variation-like penalty functionals, which are significant in reconstructing special features of solutions such as sparsity and piecewise constancy. Under certain conditions we give convergence properties of the methods. Moreover, we propose Runge-Kutta type methods to discrete the initial value problems to construct new type iterative regularization methods.
Building on the well-known total-variation (TV), this paper develops a general regularization technique based on nonlinear isotropic diffusion (NID) for inverse problems with piecewise smooth solutions. The novelty of our approach is to be adaptive (we speak of A-NID) i.e. the regularization varies during the iterates in order to incorporate prior information on the edges, deal with the evolution of the reconstruction and circumvent the limitations due to the non-convexity of the proposed functionals. After a detailed analysis of the convergence and well-posedness of the method, this latter is…
In this paper we the formulation of inverse problems as constrained minimization problems and their iterative solution by gradient or Newton type. We carry out a convergence analysis in the sense of regularization methods and discuss applicability to the problem of identifying the spatially varying diffusivity in an elliptic PDE from different sets of observations. Among these is a novel hybrid imaging techology known as impedance acoustic tomography, for which we provide numerical experiments.
The characteristic feature of inverse problems is their instability with respect to data perturbations. In order to stabilize the inversion process, regularization methods have to be developed and applied. In this work we introduce and analyze the concept of filtered diagonal frame decomposition which extends the standard filtered singular value decomposition to the frame case. Frames as generalized singular system allows to better adapt to a given class of potential solutions. In this paper, we show that filtered diagonal frame decomposition yield a convergent regularization method. Moreover,…
In this paper, we have developed an ellipsoid radial basis function neural network (ERBFNN) and algorithm for sparse representing of a molecular shape. To evaluate a sparse representation of the molecular shape model, the Gaussian density map of molecule is approximated by ERBFNN with a relatively small number of neurons. The deep learning models were trained by optimizing a nonlinear loss function with $L_1$ regularization. Experimental results demonstrate that the original molecular shape is able to be represented with good accuracy by much fewer scale of ERBFNN by our algorithm. And our net…
Regularization plays an important role in solving ill-posed problems by adding extra information about the desired solution, such as sparsity. Many regularization terms usually involve some vector norm, e.g., $L_1$ and $L_2$ norms. In this paper, we propose a novel regularization framework that uses the error function to approximate the unit step function. It can be considered as a surrogate function for the $L_0$ norm. The asymptotic behavior of the error function with respect to its intrinsic parameter indicates that the proposed regularization can approximate the standard $L_0$, $L_1$ norms…
We consider a regularization problem whose objective function consists of a convex fidelity term and a regularization term determined by the $\ell_1$ norm composed with a linear transform. Empirical results show that the regularization with the $\ell_1$ norm can promote sparsity of a regularized solution. It is the goal of this paper to understand theoretically the effect of the regularization parameter on the sparsity of the regularized solutions. We establish a characterization of the sparsity under the transform matrix of the solution. When the fidelity term has a special structure and the …
Deep learning based reconstruction methods deliver outstanding results for solving inverse problems and are therefore becoming increasingly important. A recently invented class of learning-based reconstruction methods is the so-called NETT (for Network Tikhonov Regularization), which contains a trained neural network as regularizer in generalized Tikhonov regularization. The existing analysis of NETT considers fixed operator and fixed regularizer and analyzes the convergence as the noise level in the data approaches zero. In this paper, we extend the frameworks and analysis considerably to ref…
In this paper, we develop an asymptotic expansion-regularization (AER) method for inverse source problems in two-dimensional nonlinear and nonstationary singularly perturbed partial differential equations (PDEs). The key idea of this approach is the use of the asymptotic-expansion theory, which allows us to determine the conditions for the existence and uniqueness of a solution to a given PDE with a sharp transition layer. As a by-product, we derive a simpler link equation between the source function and first-order asymptotic approximation of the measurable quantities, and based on that equat…
In this paper we extend a recent idea of formulating and regularizing inverse problems as minimization problems, so without using a forward operator, thus avoiding explicit evaluation of a parameter-to-state map. We do so by rephrasing three application examples in this minimization form, namely (a) electrical impedance tomography with the complete electrode model (b) identification of a nonlinear magnetic permeability from magnetic flux measurements (c) localization of sound sources from microphone array measurements. To establish convergence of the proposed regularization approach for these …
We introduce Stochastic Asymptotical Regularization (SAR) methods for the uncertainty quantification of the stable approximate solution of ill-posed linear-operator equations, which are deterministic models for numerous inverse problems in science and engineering. We prove the regularizing properties of SAR with regard to mean-square convergence. We also show that SAR is an optimal-order regularization method for linear ill-posed problems provided that the terminating time of SAR is chosen according to the smoothness of the solution. This result is proven for both a priori and a posteriori sto…
Institute of Biomathematics and Biometry, Helmholtz Zentrum München, German Research Center for Environmental Health（生物数学与生物计量研究所，赫尔霍兹慕尼黑中心，德国环境健康研究中心）；Technische Universität München（慕尼黑工业大学）
Variational Regularization of Inverse Problems for Manifold-Valued Data
Image Analysis and Learning Group, Interdisciplinary Center for Scientific Computing, Universität Heidelberg, Germany（图像分析与学习组，跨学科科学计算中心，海德堡大学，德国）；Department of Mathematics and Natural Sciences, Hochschule Darmstadt, and Institute of Computational Biology, Helmholtz Zentrum München, Germany（数学与自然科学系，达姆施塔特应用科学大学，以及计算生物学研究所，海德堡研究中心，德国）
We consider a family of variational regularization functionals for a generic inverse problem, where the data fidelity and regularization term are given by powers of a Hilbert norm and an absolutely one-homogeneous functional, respectively, and the regularization parameter is interpreted as artificial time. We investigate the small and large time behavior of the associated solution paths and, in particular, prove finite extinction time for a large class of functionals. Depending on the powers, we also show that the solution paths are of bounded variation or even Lipschitz continuous. In additio…
We investigate data-driven forward-inverse problems for Yajima-Oikawa (YO) system by employing two technologies which improve the performance of neural network in deep physics-informed neural network (PINN), namely neuron-wise locally adaptive activation functions and $L^2$ norm parameter regularization. Indeed, we not only recover three different forms of vector rogue waves (RWs) by means of three distinct initial-boundary value conditions in the forward problem of YO system, including bright-bright RWs, intermediate-bright RWs and dark-bright RWs, but also study the inverse problem of YO sys…
We interpret steady linear statistical inverse problems as artificial dynamic systems with white noise and introduce a stochastic differential equation (SDE) system where the inverse of the ending time $T$ naturally plays the role of the squared noise level. The time-continuous framework then allows us to apply classical methods from data assimilation, namely the Kalman-Bucy filter and 3DVAR, and to analyze their behaviour as a regularization method for the original problem. Such treatment offers some connections to the famous asymptotical regularization method, which has not yet been analyzed…
