Many inverse and parameter estimation problems can be written as PDE-constrained optimization problems. The goal, then, is to infer the parameters, typically coefficients of the PDE, from partial measurements of the solutions of the PDE for several right-hand-sides. Such PDE-constrained problems can be solved by finding a stationary point of the Lagrangian, which entails simultaneously updating the paramaters and the (adjoint) state variables. For large-scale problems, such an all-at-once approach is not feasible as it requires storing all the state variables. In this case one usually resorts to a reduced approach where the constraints are explicitly eliminated (at each iteration) by solving the PDEs. These two approaches, and variations thereof, are the main workhorses for solving PDE-constrained optimization problems arising from inverse problems. In this paper, we present an alternative method that aims to combine the advantages of both approaches. Our method is based on a quadratic penalty formulation of the constrained optimization problem. By eliminating the state variable, we develop an efficient algorithm that has roughly the same computational complexity as the conventional reduced approach while exploiting a larger search space. Numerical results show that this method indeed reduces some of the non-linearity of the problem and is less sensitive the initial iterate.
Optimal control problems including partial differential equation (PDE) as well as integer constraints merge the combinatorial difficulties of integer programming and the challenges related to large-scale systems resulting from discretized PDEs. So far, the Branch-and-Bound framework has been the most common solution strategy for such problems. In order to provide an alternative solution approach, especially in a large-scale context, this article investigates penalization techniques. Taking inspiration from a well-known family of existing exact penalty algorithms, a novel improved penalty algor…
A penalty-type method for relaxed inverse optimal control problems
University of Siegen（锡根大学）；Higher Training College, University of Yaoundé I（雅温得第一大学高等培训学院）；AIMS-Cameroon Research Center（AIMS喀麦隆研究中心）；University of Marburg（马尔堡大学）
We present a review of methods for optimal experimental design (OED) for Bayesian inverse problems governed by partial differential equations with infinite-dimensional parameters. The focus is on problems where one seeks to optimize the placement of measurement points, at which data are collected, such that the uncertainty in the estimated parameters is minimized. We present the mathematical foundations of OED in this context and survey the computational methods for the class of OED problems under study. We also outline some directions for future research in this area.
We build upon Estrin et al. (2019) to develop a general constrained nonlinear optimization algorithm based on a smooth penalty function proposed by Fletcher (1970, 1973b). Although Fletcher's approach has historically been considered impractical, we show that the computational kernels required are no more expensive than those in other widely accepted methods for nonlinear optimization. The main kernel for evaluating the penalty function and its derivatives solves structured linear systems. When the matrices are available explicitly, we store a single factorization each iteration. Otherwise, we…
We develop a general equality-constrained nonlinear optimization algorithm based on a smooth penalty function proposed by Fletcher (1970). Although it was historically considered to be computationally prohibitive in practice, we demonstrate that the computational kernels required are no more expensive than other widely accepted methods for nonlinear optimization. The main kernel required to evaluate the penalty function and its derivatives is solving a structured linear system. We show how to solve this system efficiently by storing a single factorization each iteration when the matrices are a…
Computational approaches to PDE-constrained optimization under uncertainty may involve finite-dimensional approximations of control and state spaces, sample average approximations of measures of risk and reliability, smooth approximations of nonsmooth functions, penalty approximations of constraints as well as many other kinds of inaccuracies. In this paper, we analyze the performance of controls obtained by an approximation-based algorithm and in the process develop estimates of optimality gaps for general optimization problems defined on metric spaces. Under mild assumptions, we establish th…
This work addresses optimal control problems governed by a linear time-dependent partial differential equation (PDE) as well as integer constraints on the control. Moreover, partial observations are assumed in the objective function. The resulting problem poses several numerical challenges due to the mixture of combinatorial aspects, induced by integer variables, and large scale linear algebra issues, arising from the PDE discretization. Since classical solution approaches such as the branch-and-bound framework are typically overwhelmed by such large-scale problems, this work extends an improv…
The Alternating Direction Method of Multipliers (ADMM) provides a natural way of solving inverse problems with multiple partial differential equations (PDE) forward models and nonsmooth regularization. ADMM allows splitting these large-scale inverse problems into smaller, simpler sub-problems, for which computationally efficient solvers are available. In particular, we apply large-scale second-order optimization methods to solve the fully-decoupled Tikhonov regularized inverse problems stemming from each PDE forward model. We use fast proximal methods to handle the nonsmooth regularization ter…
Decision-making problems are commonly formulated as optimization problems, which are then solved to make optimal decisions. In this work, we consider the inverse problem where we use prior decision data to uncover the underlying decision-making process in the form of a mathematical optimization model. This statistical learning problem is referred to as data-driven inverse optimization. We focus on problems where the underlying decision-making process is modeled as a convex optimization problem whose parameters are unknown. We formulate the inverse optimization problem as a bilevel program and …
BiLO: Bilevel Local Operator Learning for PDE Inverse Problems
Department of Mathematics, University of California, Irvine（加州大学伊文斯分校数学系）；Department of Computer Science, University of California, Irvine（加州大学伊文斯分校计算机科学系）；Department of Biomedical Engineering, University of California, Irvine（加州大学伊文斯分校生物医学工程系）
We present Lift and Relax for Waveform Inversion (LRWI), an approach that mitigates the local minima issue in seismic full waveform inversion (FWI) via a combination of two convexification techniques. The first technique (Lift) extends the set of variables in the optimization problem to products of those variables, arranged as a moment matrix. This algebraic idea is a celebrated way to replace a hard polynomial optimization problem by a semidefinite programming approximation. Concretely, both the model and the wavefield are lifted from vectors to rank-2 matrices. The second technique (Relax) i…
We present a method for computing A-optimal sensor placements for infinite-dimensional Bayesian linear inverse problems governed by PDEs with irreducible model uncertainties. Here, irreducible uncertainties refers to uncertainties in the model that exist in addition to the parameters in the inverse problem, and that cannot be reduced through observations. Specifically, given a statistical distribution for the model uncertainties, we compute the optimal design that minimizes the expected value of the posterior covariance trace. The expected value is discretized using Monte Carlo leading to an o…
We present an extensible software framework, hIPPYlib, for solution of large-scale deterministic and Bayesian inverse problems governed by partial differential equations (PDEs) with infinite-dimensional parameter fields (which are high-dimensional after discretization). hIPPYlib overcomes the prohibitive nature of Bayesian inversion for this class of problems by implementing state-of-the-art scalable algorithms for PDE-based inverse problems that exploit the structure of the underlying operators, notably the Hessian of the log-posterior. The key property of the algorithms implemented in hIPPYl…
We consider the ill-posed inverse problem of identifying a nonlinearity in a time-dependent PDE model. The nonlinearity is approximated by a neural network, and needs to be determined alongside other unknown physical parameters and the unknown state. Hence, it is not possible to construct input-output data pairs to perform a supervised training process. Proposing an all-at-once approach, we bypass the need for training data and recover all the unknowns simultaneously. In the general case, the approximation via a neural network can be realized as a discretization scheme, and the training with n…
We propose a fast and scalable optimization method to solve chance or probabilistic constrained optimization problems governed by partial differential equations (PDEs) with high-dimensional random parameters. To address the critical computational challenges of expensive PDE solution and high-dimensional uncertainty, we construct surrogates of the constraint function by Taylor approximation, which relies on efficient computation of the derivatives, low rank approximation of the Hessian, and a randomized algorithm for eigenvalue decomposition. To tackle the difficulty of the non-differentiabilit…
A comparative study of structural similarity and regularization for joint inverse problems governed by PDEs
Institute for Computational Engineering & Sciences, The University of Texas at Austin（德克萨斯大学奥斯汀分校计算工程与科学研究所）；Courant Institute of Mathematical Sciences, New York University（纽约大学柯朗数学科学研究所）
George Mason University（乔治梅森大学）；University of Maryland, College Park（马里兰大学帕克分校）
We consider the simulation of Bayesian statistical inverse problems governed by large-scale linear and nonlinear partial differential equations (PDEs). Markov chain Monte Carlo (MCMC) algorithms are standard techniques to solve such problems. However, MCMC techniques are computationally challenging as they require several thousands of forward PDE solves. The goal of this paper is to introduce a fractional deep neural network based approach for the forward solves within an MCMC routine. Moreover, we discuss some approximation error estimates and illustrate the efficiency of our approach via sev…
