We tackle the boundary control and estimation problems for a class of viscous Hamilton-Jacobi PDEs, considering bilateral actuation and sensing, i.e., at the two boundaries of a 1-D spatial domain. First, we solve the nonlinear trajectory generation problem for this type of PDEs, providing the necessary feedforward actions at both boundaries. Second, in order to guarantee trajectory tracking with an arbitrary decay rate, we construct nonlinear, full-state feedback laws employed at the two boundary ends. Third, a nonlinear observer is constructed, using measurements from both boundaries, which is combined with the full-state feedback designs into an observer-based output-feedback law. All of our designs are explicit since they are constructed interlacing a feedback linearizing transformation (which we introduce) with backstepping. Due to the fact that the linearizing transformation is locally invertible, only regional stability results are established, which are, nevertheless, accompanied with region of attraction estimates. Our stability proofs are based on the utilization of the linearizing transformation together with the employment of backstepping transformations, suitably formulated to handle the case of bilateral actuation and sensing. We illustrate the developed methodologies via application to traffic flow control and we present consistent simulation results.
This paper considers the backstepping state feedback control of coupled linear parabolic PDEs with spatially varying coefficients and bilateral actuation. By making use of the folding technique, a system representation with unilateral actuation is obtained, allowing to apply the standard backstepping transformation. To ensure the regularity of the solution, the folded system is subject to unusual folding boundary conditions, which lead to additional boundary couplings between the PDEs. Therefore, the solution of the corresponding kernel equations determining the transformations is a very chall…
Computing tasks may often be posed as optimization problems. The objective functions for real-world scenarios are often nonconvex and/or nondifferentiable. State-of-the-art methods for solving these problems typically only guarantee convergence to local minima. This work presents Hamilton-Jacobi-based Moreau Adaptive Descent (HJ-MAD), a zero-order algorithm with guaranteed convergence to global minima, assuming continuity of the objective function. The core idea is to compute gradients of the Moreau envelope of the objective (which is "piece-wise convex") with adaptive smoothing parameters. Gr…
We propose new and original mathematical connections between Hamilton-Jacobi (HJ) partial differential equations (PDEs) with initial data and neural network architectures. Specifically, we prove that some classes of neural networks correspond to representation formulas of HJ PDE solutions whose Hamiltonians and initial data are obtained from the parameters of the neural networks. These results do not rely on universal approximation properties of neural networks; rather, our results show that some classes of neural network architectures naturally encode the physics contained in some HJ PDEs. Ou…
Brown University（布朗大学）；Alpen-Adria Universität Klagenfurt（阿尔卑斯-亚德里亚克拉根福大学）
We consider image denoising problems formulated as variational problems. It is known that Hamilton-Jacobi PDEs govern the solution of such optimization problems when the noise model is additive. In this work, we address certain non-additive noise models and show that they are also related to Hamilton-Jacobi PDEs. These findings allow us to establish new connections between additive and non-additive noise imaging models. Specifically, we study how the solutions to these optimization problems depend on the parameters and the observed images. We show that the optimal values are ruled by some Hami…
Brown University（布朗大学）；The University of Melbourne（墨尔本大学）；UCLA（加州大学洛杉矶分校）
Solving high dimensional optimal control problems and corresponding Hamilton-Jacobi PDEs are important but challenging problems in control engineering. In this paper, we propose two abstract neural network architectures which are respectively used to compute the value function and the optimal control for certain class of high dimensional optimal control problems. We provide the mathematical analysis for the two abstract architectures. We also show several numerical results computed using the deep neural network implementations of these abstract architectures. A preliminary implementation of ou…
We develop backstepping state feedback control to stabilize a moving shockwave in a freeway segment under bilateral boundary actuations of traffic flow. A moving shockwave, consisting of light traffic upstream of the shockwave and heavy traffic downstream, is usually caused by changes of local road situations. The density discontinuity travels upstream and drivers caught in the shockwave experience transitions from free to congested traffic. Boundary control design in this paper brings the moving shockwave front to a static setpoint position, hindering the upstream propagation of traffic conge…
A deep learning approach for the approximation of the Hamilton-Jacobi-Bellman partial differential equation (HJB PDE) associated to the Nonlinear Quadratic Regulator (NLQR) problem. A state-dependent Riccati equation control law is first used to generate a gradient-augmented synthetic dataset for supervised learning. The resulting model becomes a warm start for the minimization of a loss function based on the residual of the HJB PDE. The combination of supervised learning and residual minimization avoids spurious solutions and mitigate the data inefficiency of a supervised learning-only approa…
We propose an approximation scheme for a class of semilinear parabolic equations that are convex and coercive in their gradients. Such equations arise often in pricing and portfolio management in incomplete markets and, more broadly, are directly connected to the representation of solutions to backward stochastic differential equations. The proposed scheme is based on splitting the equation in two parts, the first corresponding to a linear parabolic equation and the second to a Hamilton-Jacobi equation. The solutions of these two equations are approximated using, respectively, the Feynman-Kac …
Universität Potsdam（波茨坦大学）；Freie Universität Berlin（柏林自由大学）；Brandenburgische Technische Universität Cottbus-Senftenberg（勃兰登堡工业大学科特布斯-森夫滕贝格）
Optimal control of diffusion processes is intimately connected to the problem of solving certain Hamilton-Jacobi-Bellman equations. Building on recent machine learning inspired approaches towards high-dimensional PDEs, we investigate the potential of $\textit{iterative diffusion optimisation}$ techniques, in particular considering applications in importance sampling and rare event simulation, and focusing on problems without diffusion control, with linearly controlled drift and running costs that depend quadratically on the control. More generally, our methods apply to nonlinear parabolic PDEs…
This paper is a review of results on Optimisation which are perhaps not so standard in the PDE realm. To this end, we consider the problem of deriving the PDEs associated to the optimal control of a system of either ODEs or SDEs with respect to a vector-valued cost functional. Optimisation is considered with respect to a partial ordering generated by a given cone. Since in the vector case minima may not exist, we define vectorial value functions as (Pareto) minimals of the ordering. Our main objective is the derivation of the model PDEs which turn out to be parametric families of HJB single eq…
We propose a novel numerical method for high dimensional Hamilton--Jacobi--Bellman (HJB) type elliptic partial differential equations (PDEs). The HJB PDEs, reformulated as optimal control problems, are tackled by the actor-critic framework inspired by reinforcement learning, based on neural network parametrization of the value and control functions. Within the actor-critic framework, we employ a policy gradient approach to improve the control, while for the value function, we derive a variance reduced least-squares temporal difference method using stochastic calculus. To numerically discretize…
Hessian-augmented Supervised Learning for Hamilton-Jacobi-Bellman PDEs
Department of Mathematics, Imperial College London, South Kensington Campus（帝国理工学院伦敦数学系，南肯辛顿校区）；Department of Mathematics and Statistics, University of Konstanz（康斯坦茨大学数学与统计学系）；RICAM and Institute of Mathematics and Scientific Computing, University of Graz（格拉茨大学RICAM与数学与计算科学研究所）
The Dynamic Programming approach allows to compute a feedback control for nonlinear problems, but suffers from the curse of dimensionality. The computation of the control relies on the resolution of a nonlinear PDE, the Hamilton-Jacobi-Bellman equation, with the same dimension of the original problem. Recently, a new numerical method to compute the value function on a tree structure has been introduced. The method allows to work without a structured grid and avoids any interpolation. Here, we aim to test the algorithm for nonlinear two dimensional PDEs. We apply model order reduction to decrea…
We propose a numerical method for solving high dimensional fully nonlinear partial differential equations (PDEs). Our algorithm estimates simultaneously by backward time induction the solution and its gradient by multi-layer neural networks, while the Hessian is approximated by automatic differentiation of the gradient at previous step. This methodology extends to the fully nonlinear case the approach recently proposed in \cite{HPW19} for semi-linear PDEs. Numerical tests illustrate the performance and accuracy of our method on several examples in high dimension with nonlinearity on the Hessia…
In this paper, we consider nonlinear PDEs in a port-Hamiltonian setting based on an underlying jet-bundle structure. We restrict ourselves to systems with 1-dimensional spatial domain and 2nd-order Hamiltonian including certain dissipation models that can be incorporated in the port- Hamiltonian framework by means of appropriate differential operators. For this system class, energy-based control by means of Casimir functionals as well as energy balancing is analysed and demonstrated using a nonlinear Euler-Bernoulli beam.
