This paper presents a real-time optimization method for nonlinear model predictive control (NMPC) of systems governed by partial differential equations (PDEs). The NMPC problem to be solved is formulated by discretizing the PDE system in space and time by using the finite difference method. The proposed method is called the double-layer Jacobi method, which exploits both the spatial and temporal sparsities of the PDE-constrained NMPC problem. In the upper layer, the NMPC problem is solved by ignoring the temporal couplings of either the state or costate (Lagrange multiplier corresponding to the state equation) equations so that the spatial sparsity is preserved. The lower-layer Jacobi method is a linear solver dedicated to PDE-constrained NMPC problems by exploiting the spatial sparsity. Convergence analysis indicates that the convergence of the proposed method is related to the prediction horizon and regularization. Results of a numerical experiment of controlling a heat transfer process show that the proposed method is two orders of magnitude faster than the conventional structure-exploiting Newton's method.
A deep learning approach for the approximation of the Hamilton-Jacobi-Bellman partial differential equation (HJB PDE) associated to the Nonlinear Quadratic Regulator (NLQR) problem. A state-dependent Riccati equation control law is first used to generate a gradient-augmented synthetic dataset for supervised learning. The resulting model becomes a warm start for the minimization of a loss function based on the residual of the HJB PDE. The combination of supervised learning and residual minimization avoids spurious solutions and mitigate the data inefficiency of a supervised learning-only approa…
Brown University（布朗大学）；Alpen-Adria Universität Klagenfurt（阿尔卑斯-亚德里亚克拉根福大学）
We consider image denoising problems formulated as variational problems. It is known that Hamilton-Jacobi PDEs govern the solution of such optimization problems when the noise model is additive. In this work, we address certain non-additive noise models and show that they are also related to Hamilton-Jacobi PDEs. These findings allow us to establish new connections between additive and non-additive noise imaging models. Specifically, we study how the solutions to these optimization problems depend on the parameters and the observed images. We show that the optimal values are ruled by some Hami…
We present methods for locally solving the Dynamic Programming Equations (DPE) and the Hamilton Jacobi Bellman (HJB) PDE that arise in the infinite horizon optimal control problem. The method for solving the DPE is the discrete time version of Al'brecht's procedure for locally approximating the solution of the HJB. We also prove the existence of the smooth solutions to the DPE that has the same Taylor series expansions as the formal solutions. Our procedure for solving the HJB PDE numerically begins with Al'brecht's local solution as the initial approximation and uses some Lyapunov criteria to…
Computing tasks may often be posed as optimization problems. The objective functions for real-world scenarios are often nonconvex and/or nondifferentiable. State-of-the-art methods for solving these problems typically only guarantee convergence to local minima. This work presents Hamilton-Jacobi-based Moreau Adaptive Descent (HJ-MAD), a zero-order algorithm with guaranteed convergence to global minima, assuming continuity of the objective function. The core idea is to compute gradients of the Moreau envelope of the objective (which is "piece-wise convex") with adaptive smoothing parameters. Gr…
Brown University（布朗大学）；The University of Melbourne（墨尔本大学）；UCLA（加州大学洛杉矶分校）
Solving high dimensional optimal control problems and corresponding Hamilton-Jacobi PDEs are important but challenging problems in control engineering. In this paper, we propose two abstract neural network architectures which are respectively used to compute the value function and the optimal control for certain class of high dimensional optimal control problems. We provide the mathematical analysis for the two abstract architectures. We also show several numerical results computed using the deep neural network implementations of these abstract architectures. A preliminary implementation of ou…
We propose new and original mathematical connections between Hamilton-Jacobi (HJ) partial differential equations (PDEs) with initial data and neural network architectures. Specifically, we prove that some classes of neural networks correspond to representation formulas of HJ PDE solutions whose Hamiltonians and initial data are obtained from the parameters of the neural networks. These results do not rely on universal approximation properties of neural networks; rather, our results show that some classes of neural network architectures naturally encode the physics contained in some HJ PDEs. Ou…
This paper is a review of results on Optimisation which are perhaps not so standard in the PDE realm. To this end, we consider the problem of deriving the PDEs associated to the optimal control of a system of either ODEs or SDEs with respect to a vector-valued cost functional. Optimisation is considered with respect to a partial ordering generated by a given cone. Since in the vector case minima may not exist, we define vectorial value functions as (Pareto) minimals of the ordering. Our main objective is the derivation of the model PDEs which turn out to be parametric families of HJB single eq…
We tackle the boundary control and estimation problems for a class of viscous Hamilton-Jacobi PDEs, considering bilateral actuation and sensing, i.e., at the two boundaries of a 1-D spatial domain. First, we solve the nonlinear trajectory generation problem for this type of PDEs, providing the necessary feedforward actions at both boundaries. Second, in order to guarantee trajectory tracking with an arbitrary decay rate, we construct nonlinear, full-state feedback laws employed at the two boundary ends. Third, a nonlinear observer is constructed, using measurements from both boundaries, which …
Universität Potsdam（波茨坦大学）；Freie Universität Berlin（柏林自由大学）；Brandenburgische Technische Universität Cottbus-Senftenberg（勃兰登堡工业大学科特布斯-森夫滕贝格）
Optimal control of diffusion processes is intimately connected to the problem of solving certain Hamilton-Jacobi-Bellman equations. Building on recent machine learning inspired approaches towards high-dimensional PDEs, we investigate the potential of $\textit{iterative diffusion optimisation}$ techniques, in particular considering applications in importance sampling and rare event simulation, and focusing on problems without diffusion control, with linearly controlled drift and running costs that depend quadratically on the control. More generally, our methods apply to nonlinear parabolic PDEs…
We consider the optimal control of solutions of first order Hamilton-Jacobi equations, where the Hamiltonian is convex with linear growth. This models the problem of steering the propagation of a front by constructing an obstacle. We prove existence of minimizers to this optimization problem as in a relaxed setting and characterize the minimizers as weak solutions to a mean field game type system of coupled partial differential equations. Furthermore, we prove existence and partial uniqueness of weak solutions to the PDE system. An interpretation in terms of mean field games is also discussed.…
We present a control design for semilinear and quasilinear 2x2 hyperbolic partial differential equations with the control input at one boundary and a nonlinear ordinary differential equation coupled to the other. The controller can be designed to asymptotically stabilize the system at an equilibrium or relative to a reference signal. Two related but different controllers for semilinear and general quasilinear systems are presented and the additional challenges in quasilinear systems are discussed. Moreover, we present an observer that estimates the distributed PDE state and the unmeasured ODE …
We show how a posteriori goal oriented error estimation can be used to efficiently solve the subproblems occurring in a Model Predictive Control (MPC) algorithm. In MPC, only an initial part of a computed solution is implemented as a feedback, which motivates grid refinement particularly tailored to this context. To this end, we present a truncated cost functional as objective for goal oriented adaptivity and prove under stabilizability assumptions that error indicators decay exponentially outside the support of this quantity. This leads to very efficient time and space discretizations for MPC…
We propose a novel numerical method for high dimensional Hamilton--Jacobi--Bellman (HJB) type elliptic partial differential equations (PDEs). The HJB PDEs, reformulated as optimal control problems, are tackled by the actor-critic framework inspired by reinforcement learning, based on neural network parametrization of the value and control functions. Within the actor-critic framework, we employ a policy gradient approach to improve the control, while for the value function, we derive a variance reduced least-squares temporal difference method using stochastic calculus. To numerically discretize…
It is well known that time dependent Hamilton-Jacobi-Isaacs partial differential equations (HJ PDE), play an important role in analyzing continuous dynamic games and control theory problems. An important tool for such problems when they involve geometric motion is the level set method. This was first used for reachability problems. The cost of these algorithms, and, in fact, all PDE numerical approximations is exponential in the space dimension and time. In this work we propose and test methods for solving a large class of the HJ PDE relevant to optimal control problems without the use of grid…
In this paper we introduce an iterative Jacobi algorithm for solving distributed model predictive control (DMPC) problems, with linear coupled dynamics and convex coupled constraints. The algorithm guarantees stability and persistent feasibility, and we provide a localized procedure for constructing an initial feasible solution by constraint tightening. Moreover, we show that the solution of the iterative process converges to the centralized MPC solution. The proposed iterative approach involves solving local optimization problems consisting of only few subsystems, depending on the choice of t…
Hessian-augmented Supervised Learning for Hamilton-Jacobi-Bellman PDEs
Department of Mathematics, Imperial College London, South Kensington Campus（帝国理工学院伦敦数学系，南肯辛顿校区）；Department of Mathematics and Statistics, University of Konstanz（康斯坦茨大学数学与统计学系）；RICAM and Institute of Mathematics and Scientific Computing, University of Graz（格拉茨大学RICAM与数学与计算科学研究所）
The classical inward pointing condition (IPC) for a control system whose state $x$ is constrained in the closure $C:=\barΩ$ of an open set $Ω$ prescribes that at each point of the boundary $x\in \partial Ω$ the intersection between the dynamics and the interior of the tangent space of $\bar Ω$ at $x$ is nonempty. Under this hypothesis, for every system trajectory $x(.)$ on a time-interval $[0,T]$, possibly violating the constraint, one can construct a new system trajectory $\hat x(.)$ that satisfies the constraint and whose distance from $x(.)$ is bounded by a quantity proportional to the maxi…
In a previous work (Akian, Fodjo, 2016), we introduced a lower complexity probabilistic max-plus numerical method for solving fully nonlinear Hamilton-Jacobi-Bellman equations associated to diffusion control problems involving a finite set-valued (or switching) control and possibly a continuum-valued control. This method was based on the idempotent expansion properties obtained by McEneaney, Kaise and Han (2011) and on the numerical probabilistic method proposed by Fahim, Touzi and Warin (2011) for solving some fully nonlinear parabolic partial differential equations. A difficulty of the algor…
In the square root velocity framework, the computation of shape space distances and the registration of curves requires solution of a non-convex variational problem. In this paper, we present a new PDE-based method for solving this problem numerically. The method is constructed from numerical approximation of the Hamilton-Jacobi-Bellman equation for the variational problem, and has quadratic complexity and global convergence for the distance estimate. In conjunction, we propose a backtracking scheme for approximating solutions of the registration problem, which additionally can be used to comp…
Policy iteration for Hamilton-Jacobi-Isaacs equations with control constraints and comparison with Hamilton-Jacobi-Bellman equations
Rajiv Gandhi Institute of Petroleum Technology（拉吉夫·甘地石油技术学院）；University of Graz（格拉茨大学）；Radon Institute for Computational and Applied Mathematics (RICAM)（雷登计算与应用数学研究所）
In this paper, we develop algorithms to overcome the curse of dimensionality in possibly non-convex state-dependent Hamilton-Jacobi equations (HJ PDEs) arising from optimal control and differential game problems. The subproblems are independent and can be implemented in an embarrassingly parallel fashion. This is an ideal setup for perfect scaling in parallel computing. The algorithm is proposed to overcome the curse of dimensionality [1, 2] when solving HJ PDE. The major contribution of the paper is to change an optimization problem over a space of curves to an optimization problem of a singl…
The Dynamic Programming approach allows to compute a feedback control for nonlinear problems, but suffers from the curse of dimensionality. The computation of the control relies on the resolution of a nonlinear PDE, the Hamilton-Jacobi-Bellman equation, with the same dimension of the original problem. Recently, a new numerical method to compute the value function on a tree structure has been introduced. The method allows to work without a structured grid and avoids any interpolation. Here, we aim to test the algorithm for nonlinear two dimensional PDEs. We apply model order reduction to decrea…
We propose a numerical method for solving high dimensional fully nonlinear partial differential equations (PDEs). Our algorithm estimates simultaneously by backward time induction the solution and its gradient by multi-layer neural networks, while the Hessian is approximated by automatic differentiation of the gradient at previous step. This methodology extends to the fully nonlinear case the approach recently proposed in \cite{HPW19} for semi-linear PDEs. Numerical tests illustrate the performance and accuracy of our method on several examples in high dimension with nonlinearity on the Hessia…
Systems of Hamilton-Jacobi equations arise naturally when we study the optimal control problems with pathwise deterministic trajectories with random switching. In this work, we are interested in the large time behavior of weakly coupled systems of first-order Hamilton-Jacobi equations in the periodic setting. The large time behavior for systems of Hamilton-Jacobi equations have been obtained by Camilli-Loreti-Ley and the author (2012) and Mitake-Tran (2012) under quite strict conditions. In this work, we use a PDE approach to extend the convergence result proved by Barles-Souganidis (2000) in …
In the development of model predictive controllers for PDE-constrained problems, the use of reduced order models is essential to enable real-time applicability. Besides local linearization approaches, Proper Orthogonal Decomposition (POD) has been most widely used in the past in order to derive such models. Due to the huge advances concerning both theory as well as the numerical approximation, a very promising alternative based on the Koopman operator has recently emerged. In this chapter, we present two control strategies for model predictive control of nonlinear PDEs using data-efficient app…
We provide a new approach for the efficient matrix-free application of the transpose of the Jacobian for the spectral element method for the adjoint based solution of partial differential equation (PDE) constrained optimization. This results in optimizations of nonlinear PDEs using explicit integrators where the integration of the adjoint problem is not more expensive than the forward simulation. Solving PDE constrained optimization problems entails combining expertise from multiple areas, including simulation, computation of derivatives, and optimization. The Portable, Extensible Toolkit for …
University of California, Los Angeles（加利福尼亚大学洛杉矶分校）；McGill University（麦吉尔大学）；Université Paris IX Dauphine（巴黎第九大学）
In this paper we establish a connection between non-convex optimization methods for training deep neural networks and nonlinear partial differential equations (PDEs). Relaxation techniques arising in statistical physics which have already been used successfully in this context are reinterpreted as solutions of a viscous Hamilton-Jacobi PDE. Using a stochastic control interpretation allows we prove that the modified algorithm performs better in expectation that stochastic gradient descent. Well-known PDE regularity results allow us to analyze the geometry of the relaxed energy landscape, confir…
Bayesian inference provides a systematic framework for integration of data with mathematical models to quantify the uncertainty in the solution of the inverse problem. However, the solution of Bayesian inverse problems governed by complex forward models described by partial differential equations (PDEs) remains prohibitive with black-box Markov chain Monte Carlo (MCMC) methods. We present hIPPYlib-MUQ, an extensible and scalable software framework that contains implementations of state-of-the art algorithms aimed to overcome the challenges of high-dimensional, PDE-constrained Bayesian inverse …
We propose efficient numerical schemes for implementing the natural gradient descent (NGD) for a broad range of metric spaces with applications to PDE-based optimization problems. Our technique represents the natural gradient direction as a solution to a standard least-squares problem. Hence, instead of calculating, storing, or inverting the information matrix directly, we apply efficient methods from numerical linear algebra. We treat both scenarios where the Jacobian, i.e., the derivative of the state variable with respect to the parameter, is either explicitly known or implicitly given thro…
