The Bayesian approach to inverse problems is widely used in practice to infer unknown parameters from noisy observations. In this framework, the ensemble Kalman inversion has been successfully applied for the quantification of uncertainties in various areas of applications. In recent years, a complete analysis of the method has been developed for linear inverse problems adopting an optimization viewpoint. However, many applications require the incorporation of additional constraints on the parameters, e.g. arising due to physical constraints. We propose a new variant of the ensemble Kalman inversion to include box constraints on the unknown parameters motivated by the theory of projected preconditioned gradient flows. Based on the continuous time limit of the constrained ensemble Kalman inversion, we discuss a complete convergence analysis for linear forward problems. We adopt techniques from filtering which are crucial in order to improve the performance and establish a correct descent, such as variance inflation. These benefits are highlighted through a number of numerical examples on various inverse problems based on partial differential equations.
Ensemble Kalman inversion (EKI) is a derivative-free optimization method that lies between the deterministic and the probabilistic approaches for inverse problems. EKI iterates the Kalman update of ensemble-based Kalman filters, whose ensemble converges to a minimizer of an objective function. EKI regularizes ill-posed problems by restricting the ensemble to the linear span of the initial ensemble, or by iterating regularization with early stopping. Another regularization approach for EKI, Tikhonov EKI, penalizes the objective function using the $l_2$ penalty term, preventing overfitting in th…
Ensemble Kalman inversion is a parallelizable derivative-free method to solve inverse problems. The method uses an ensemble that follows the Kalman update formula iteratively to solve an optimization problem. The ensemble size is crucial to capture the correct statistical information in estimating the unknown variable of interest. Still, the ensemble is limited to a size smaller than the unknown variable's dimension for computational efficiency. This study proposes a strategy to correct the sampling error due to a small ensemble size, which improves the performance of the ensemble Kalman inver…
Many data-science problems can be formulated as an inverse problem, where the parameters are estimated by minimizing a proper loss function. When complicated black-box models are involved, derivative-free optimization tools are often needed. The ensemble Kalman filter (EnKF) is a particle-based derivative-free Bayesian algorithm originally designed for data assimilation. Recently, it has been applied to inverse problems for computational efficiency. The resulting algorithm, known as ensemble Kalman inversion (EKI), involves running an ensemble of particles with EnKF update rules so they can co…
We study the use of novel techniques arising in machine learning for inverse problems. Our approach replaces the complex forward model by a neural network, which is trained simultaneously in a one-shot sense when estimating the unknown parameters from data, i.e. the neural network is trained only for the unknown parameter. By establishing a link to the Bayesian approach to inverse problems, an algorithmic framework is developed which ensures the feasibility of the parameter estimate w.r. to the forward model. We propose an efficient, derivative-free optimization method based on variants of the…
The Ensemble Kalman Filter (EnKF) belongs to the class of iterative particle filtering methods and can be used for solving control--to--observable inverse problems. In this context, the EnKF is known as Ensemble Kalman Inversion (EKI). In recent years several continuous limits in the number of iteration and particles have been performed in order to study properties of the method. In particular, a one--dimensional linear stability analysis reveals possible drawbacks in the phase space of moments provided by the continuous limits of the EKI, but observed also in the multi--dimensional setting. I…
The unscented Kalman inversion (UKI) presented in [1] is a general derivative-free approach to solving the inverse problem. UKI is particularly suitable for inverse problems where the forward model is given as a black box and may not be differentiable. The regularization strategy and convergence property of the UKI are thoroughly studied, and the method is demonstrated effectively handling noisy observation data and solving chaotic inverse problems. In this paper, we aim to make the UKI more efficient in terms of computational and memory costs for large scale inverse problems. We take advantag…
The Ensemble Kalman inversion (EKI) method is a method for the estimation of unknown parameters in the context of (Bayesian) inverse problems. The method approximates the underlying measure by an ensemble of particles and iteratively applies the ensemble Kalman update to evolve (the approximation of the) prior into the posterior measure. For the convergence analysis of the EKI it is common practice to derive a continuous version, replacing the iteration with a stochastic differential equation. In this paper we validate this approach by showing that the stochastic EKI iteration converges to pat…
We propose a new regularisation strategy for the classical ensemble Kalman inversion (EKI) framework. The strategy consists of: (i) an adaptive choice for the regularisation parameter in the update formula in EKI, and (ii) criteria for the early stopping of the scheme. In contrast to existing approaches, our parameter choice does not rely on additional tuning parameters which often have severe effects on the efficiency of EKI. We motivate our approach using the interpretation of EKI as a Gaussian approximation in the Bayesian tempering setting for inverse problems. We show that our parameter c…
Convergence Analysis of Ensemble Kalman Inversion: The Linear, Noisy Case
Institute for Mathematics, University of Mannheim（曼海姆大学数学研究所）；Department of Computing and Mathematical Sciences, California Institute of Technology（加州理工学院计算与数学科学系）
The ensemble Kalman inversion (EKI) is a particle based method which has been introduced as the application of the ensemble Kalman filter to inverse problems. In practice it has been widely used as derivative-free optimization method in order to estimate unknown parameters from noisy measurement data. For linear forward models the EKI can be viewed as gradient flow preconditioned by a certain sample covariance matrix. Through the preconditioning the resulting scheme remains in a finite dimensional subspace of the original high-dimensional (or even infinite dimensional) parameter space and can …
Ensemble Kalman Inversion as an Inertial Interacting Particle System
Institute for Geometry and Applied Mathematics, RWTH Aachen University（亚琛工业大学几何与应用数学研究所）；Department of Mathematics and Applied Mathematics, University of Pretoria（比勒陀利亚大学数学与应用数学系）；Department of Mathematics, Sapienza University of Rome（罗马第一大学数学系）
Ensemble Kalman inversion (EKI) is a derivative-free optimizer aimed at solving inverse problems, taking motivation from the celebrated ensemble Kalman filter. The purpose of this article is to consider the introduction of adaptive Tikhonov strategies for EKI. This work builds upon Tikhonov EKI (TEKI) which was proposed for a fixed regularization constant. By adaptively learning the regularization parameter, this procedure is known to improve the recovery of the underlying unknown. For the analysis, we consider a continuous-time setting where we extend known results such as well-posdeness and …
The unscented Kalman inversion (UKI) method presented in [1] is a general derivative-free approach for the inverse problem. UKI is particularly suitable for inverse problems where the forward model is given as a black box and may not be differentiable. The regularization strategies, convergence property, and speed-up strategies [1,2] of the UKI are thoroughly studied, and the method is capable of handling noisy observation data and solving chaotic inverse problems. In this paper, we study the uncertainty quantification capability of the UKI. We propose a modified UKI, which allows to well appr…
The interaction between the foundation structures and the soil has been developed for many engineering applications. For the determination of the stress in foundation structure it is needed to determine the influence of the stiffness of soil with respect to the displacement w of the deformable plate (direct problem), and viceversa, how the stiffness of the foundation structure affects the resulting subsidence (inverse problem). In this paper, we deal with the Winkler mathematical model and propose to use an efficient Ensemble Kalman Inversion scheme (EKI) that regularizes iteratively the ill-p…
Mathematical modeling and simulation of complex physical systems based on partial differential equations (PDEs) have been widely used in engineering and industrial applications. To enable reliable predictions, it is crucial yet challenging to calibrate the model by inferring unknown parameters/fields (e.g., boundary conditions, mechanical properties, and operating parameters) from sparse and noisy measurements, which is known as a PDE-constrained inverse problem. In this work, we develop a novel bi-fidelity (BF) ensemble Kalman inversion method to tackle this challenge, leveraging the accuracy…
The ensemble Kalman inversion (EKI) for the solution of Bayesian inverse problems of type $y = A u +\varepsilon$, with $u$ being an unknown parameter, $y$ a given datum, and $\varepsilon$ measurement noise, is a powerful tool usually derived from a sequential Monte Carlo point of view. It describes the dynamics of an ensemble of particles $\{u^j(t)\}_{j=1}^J$, whose initial empirical measure is sampled from the prior, evolving over an artificial time $t$ towards an approximate solution of the inverse problem, with $t=1$ emulating the posterior, and $t\to\infty$ corresponding to the under-regul…
The spatial dependent unknown acoustic source is reconstructed according noisy multiple frequency data on a remote closed surface. Assume that the unknown function is supported on a bounded domain. To determine the support, we present a statistical inversion algorithm, which combines the ensemble Kalman filter approach with level set technique. Several numerical examples show that the proposed method give good numerical reconstruction.
This paper is focused on the optimization approach to the solution of inverse problems. We introduce a stochastic dynamical system in which the parameter-to-data map is embedded, with the goal of employing techniques from nonlinear Kalman filtering to estimate the parameter given the data. The extended Kalman filter (which we refer to as ExKI in the context of inverse problems) can be effective for some inverse problems approached this way, but is impractical when the forward map is not readily differentiable and is given as a black box, and also for high dimensional parameter spaces because o…
The ensemble Kalman filter is a well-known and celebrated data assimilation algorithm. It is of particular relevance as it used for high-dimensional problems, by updating an ensemble of particles through a sample mean and covariance matrices. In this chapter we present a relatively recent topic which is the application of the EnKF to inverse problems, known as ensemble Kalman Inversion (EKI). EKI is used for parameter estimation, which can be viewed as a black-box optimizer for PDE-constrained inverse problems. We present in this chapter a review of the discussed methodology, while presenting …
Inverse problems occur in a variety of parameter identification tasks in engineering. Such problems are challenging in practice, as they require repeated evaluation of computationally expensive forward models. We introduce a unifying framework of multilevel optimization that can be applied to a wide range of optimization-based solvers. Our framework provably reduces the computational cost associated with evaluating the expensive forward maps stemming from various physical models. To demonstrate the versatility of our analysis, we discuss its implications for various methodologies including mul…
Often, when solving forward, inverse or data assimilation problems, only a part of the solution is needed. As a model, we consider the stationary diffusion problem. We demonstrate an algorithm that can compute only a part or a functional of the solution, without calculating the full inversion operator and the complete solution. It is a well-known fact about partial differential equations that the solution at each discretisation point depends on the solutions at all other discretisation points. Therefore, it is impossible to compute the solution only at one point, without calculating the soluti…
The prediction of crack initiation and propagation in ductile failure processes are challenging tasks for the design and fabrication of metallic materials and structures on a large scale. Numerical aspects of ductile failure dictate a sub-optimal calibration of plasticity- and fracture-related parameters for a large number of material properties. These parameters enter the system of partial differential equations as a forward model. In this work, we develop a step-wise Bayesian inversion framework for ductile fracture to provide accurate knowledge regarding the effective mechanical parameters.…
Bayesian calibration is widely used for inverse analysis and uncertainty analysis for complex systems in the presence of both computer models and observation data. In the present work, we focus on large-scale fluid-structure interaction systems characterized by large structural deformations. Numerical methods to solve these problems, including embedded/immersed boundary methods, are typically not differentiable and lack smoothness. We propose a framework that is built on unscented Kalman filter/inversion to efficiently calibrate and provide uncertainty estimations of such complicated models wi…
