We obtain weighted uniform estimates for the gradient of the solutions to a class of linear parabolic Cauchy problems with unbounded coefficients. Such estimates are then used to prove existence and uniqueness of the mild solution to a semi-linear backward parabolic Cauchy problem, where the differential equation is the Hamilton-Jacobi-Bellman equation of a suitable optimal control problem. Via backward stochastic differential equations, we show that the mild solution is indeed the Value Function of the controlled equation and that the feedback law is verified.
This paper is concerned with the asymptotic analysis of infinite systems of weakly coupled stationary Hamilton-Jacobi-Bellman equations as the discount factor tends to zero. With a specific Hamiltonian, we show the convergence of the solution and prove the solvability of the corresponding ergodic problem.
In this paper we provide a rate of convergence for periodic homogenization of Hamilton-Jacobi-Bellman equations with nonlocal diffusion. The result is based on the regularity of the associated effective problem, where the convexity plays a crucial role. Such regularity estimates are possible from the available literature once we provide a representation formula for the effective Hamiltonian, a result that has an independent interest.
This paper concerns continuous dependence estimates for Hamilton-Jacobi-Bellman-Isaacs operators (briefly, HJBI). For the parabolic Cauchy problem, we establish such an estimate in the whole space $[0,+\infty)\times\Rn$. Moreover, under some periodicity and ellipticity assumptions, we obtain a similar estimate for the ergodic constant associated to the HJBI operator. An interesting byproduct of the latter result will be the local uniform convergence for some classes of singular perturbation problems.
We study the partial differential equation max{Lu - f, H(Du)}=0 where u is the unknown function, L is a second-order elliptic operator, f is a given smooth function and H is a convex function. This is a model equation for Hamilton-Jacobi-Bellman equations arising in stochastic singular control. We establish the existence of a unique viscosity solution of the Dirichlet problem that has a Holder continuous gradient. We also show that if H is uniformly convex, the gradient of this solution is Lipschitz continuous.
In this paper we establish periodic homogenization for Hamilton-Jacobi-Bellman (HJB) equations, associated to nonlocal operators of integro-differential type. We consider the case when the fractional diffusion has the same order as the drift term, and is weakly elliptic. The outcome of the paper is twofold. One one hand, we provide Lipschitz regularity results for weakly elliptic non-local HJB, extending the results previously obtained in [8]. On the other hand, we establish a convergence result, based on half relaxed limits and a comparison principle for the effective problem. The latter stro…
The quasi-periodic homogenization for some classes of first-order Hamilton-Jaconi-Bellman equation is studied in this paper. The cell problem of the quasi-periodic homogenization satisfies the non-resonance condition, under which the corresponding deterministic system is ergodic. The almost periodic homogenization for the same classes of equations is also solved, as a limit of a sequence of quasi-periodic homogenizations. Here, the almost periodicity is in the sense of H. Bohr. This result has been cited by some authors, for example: by H. Ishii, "Almost periodic homogenization of Hamilton-Jac…
Here, we study the large-time limit of viscosity solutions of the Cauchy problem for second-order Hamilton--Jacobi--Bellman equations with convex Hamiltonians in the torus. This large-time limit solves the corresponding stationary problem, sometimes called the ergodic problem. This problem, however, has multiple viscosity solutions and, thus, a key question is which of these solutions is selected by the limit. Here, we provide a representation for the viscosity solution to the Cauchy problem in terms of generalized holonomic measures. Then, we use this representation to characterize the large-…
We derive the long time asymptotic of solutions to an evolutive Hamilton-Jacobi-Bellman equation in a bounded smooth domain, in connection with ergodic problems recently studied in \cite{bcr}. Our main assumption is an appropriate degeneracy condition on the operator at the boundary. This condition is related to the characteristic boundary points for linear operators as well as to the irrelevant points for the generalized Dirichlet problem, and implies in particular that no boundary datum has to be imposed. We prove that there exists a constant $c$ such that the solutions of the evolutive prob…
We study PDE of the form $\max\{F(D^2u,x)-f(x), H(Du)\}=0$ where $F$ is uniformly elliptic and convex in its first argument, $H$ is convex, $f$ is a given function and $u$ is the unknown. These equations are derived from dynamic programming in a wide class of stochastic singular control problems. In particular, examples of these equations arise in mathematical finance models involving transaction costs, in queuing theory, and spacecraft control problems. The main aspects of this work are to identify conditions under which solutions are uniquely defined and have Lipschitz continuous gradients. …
The aim of this paper is to construct and analyze solutions to a class of Hamilton-Jacobi-Bellman equations with range bounds on the optimal response variable. Using the Riccati transformation we derive and analyze a fully nonlinear parabolic partial differential equation for the optimal response function. We construct monotone traveling wave solutions and identify parametric regions for which the traveling wave solution has a positive or negative wave speed.
In this paper we conduct a priori and a posteriori error analysis of the $C^0$ interior penalty method for Hamilton-Jacobi-Bellman equations, with coefficients that satisfy the Cordes condition. These estimates show the quasi-optimality of the method, and provide one with an adaptive finite element method. In accordance with the proven regularity theory, we only assume that the solution of the Hamilton-Jacobi-Bellman equation belongs to $H^2$.
In this paper we show a uniqueness result for weak epigraphical solutions of Hamilton-Jacobi-Bellman (HJB) equations on infinite horizon for a class of lower semicontinuous functions vanishing at infinity. Weak epigraphical solutions of HJB equations, with time-measurable data and fiber-convex, turn out to be viscosity solutions - in the classical sense - whenever they are locally Lipschitz continuous. Here we extend the notion of locally absolutely continuous tubes to set-valued maps with continuous epigraph of locally bounded variations. This new notion fits with the lack of uniform lower bo…
We study in this series of articles the Kardar-Parisi-Zhang (KPZ) equation $$ \partial_t h(t,x)=νΔh(t,x)+λV(|\nabla h(t,x)|) +\sqrt{D}\, η(t,x), \qquad x\in{\mathbb{R}}^d $$ in $d\ge 1$ dimensions. The forcing term $η$ in the right-hand side is a regularized white noise. The deposition rate $V$ is assumed to be isotropic and convex. Assuming $V(0)\ge 0$, one finds $V(|\nabla h|)\ltimes |\nabla h|^2$ for small gradients, yielding the equation which is most commonly used in the literature. The present article, a continuation of [24], is dedicated to a generalization of the PDE estimates obtained…
We study the well-posedness of Hamilton-Jacobi-Bellman equations on subsets of $\mathbb{R}^d$ in a context without boundary conditions. The Hamiltonian is given as the supremum over two parts: an internal Hamiltonian depending on an external control variable and a cost functional penalizing the control. The key feature in this paper is that the control function can be unbounded and discontinuous. This way we can treat functionals that appear e.g. in the Donsker-Varadhan theory of large deviations for occupation time measures. To allow for this flexibility, we assume that the internal Hamiltoni…
We study the representation formulae for the fundamental solutions and viscosity solutions of the Hamilton-Jacobi equations of contact type. We also obtain a vanishing contact structure result for relevant Cauchy problems which can be regarded as an extension to the vanishing discount problem.
In this paper we study the fully nonlinear stochastic Hamilton-Jacobi-Bellman (HJB) equation for the optimal stochastic control problem of stochastic differential equations with random coefficients. The notion of viscosity solution is introduced, and we prove that the value function of the optimal stochastic control problem is the maximal viscosity solution of the associated stochastic HJB equation. For the superparabolic cases when the diffusion coefficients are deterministic functions of time, states and controls, the uniqueness is addressed as well.
We consider the homogenization of Hamilton-Jacobi equations and degenerate Bellman equations in stationary, ergodic, unbounded environments. We prove that, as the microscopic scale tends to zero, the equation averages to a deterministic Hamilton-Jacobi equation and study some properties of the effective Hamiltonian. We discover a connection between the effective Hamiltonian and an eikonal-type equation in exterior domains. In particular, we obtain a new formula for the effective Hamiltonian. To prove the results we introduce a new strategy to obtain almost sure homogenization, completing a pro…
In this paper, we prove a comparison result between semicontinuous viscosity sub and supersolutions growing at most quadratically of second-order degenerate parabolic Hamilton-Jacobi-Bellman and Isaacs equations. As an application, we characterize the value function of a finite horizon stochastic control problem with unbounded controls as the unique viscosity solution of the corresponding dynamic programming equation.
We analyze the consequences that the so-called turnpike property has on the long-time behavior of the value function corresponding to a finite-dimensional linear-quadratic optimal control problem with general terminal cost and constrained controls. We prove that, when the time horizon $T$ tends to infinity, the value function asymptotically behaves as $W(x) + c\, T + λ$, and we provide a control interpretation of each of these three terms, making clear the link with the turnpike property. As a by-product, we obtain the long-time behavior of the solution to the associated Hamilton-Jacobi-Bellma…
We study the regularity properties of integro-partial differential equations of Hamilton-Jocobi-Bellman type with terminal condition, which can be interpreted through a stochastic control system, composed of a forward and a backward stochastic differential equation, both driven by a Brownian motion and a compensated Poisson random measure. More precisely, we prove that, under appropriate assumptions, the viscosity solution of such equations is jointly Lipschitz and jointly semiconcave in $(t,x)\inΔ\times\R^d$, for all compact time intervals $Δ$ excluding the terminal time. Our approach is base…
We consider a Bolza-type optimal control problem for a dynamical system described by a fractional differential equation with the Caputo derivative of an order $α\in (0, 1)$. The value of this problem is introduced as a functional in a suitable space of histories of motions. We prove that this functional satisfies the dynamic programming principle. Based on a new notion of coinvariant derivatives of the order $α$, we associate the considered optimal control problem with a Hamilton-Jacobi-Bellman equation. Under certain smoothness assumptions, we establish a connection between the value function…
This paper introduces the Hamilton-Jacobi-Bellman Proximal Policy Optimization (HJBPPO) algorithm into reinforcement learning. The Hamilton-Jacobi-Bellman (HJB) equation is used in control theory to evaluate the optimality of the value function. Our work combines the HJB equation with reinforcement learning in continuous state and action spaces to improve the training of the value network. We treat the value network as a Physics-Informed Neural Network (PINN) to solve for the HJB equation by computing its derivatives with respect to its inputs exactly. The Proximal Policy Optimization (PPO)-Cl…
In this paper, we propose and study the stochastic path-dependent Hamilton-Jacobi-Bellman (SPHJB) equation that arises naturally from the optimal stochastic control problem of stochastic differential equations with path-dependence and measurable randomness. Both the notions of viscosity solution and classical solution are proposed, and the value function of the optimal stochastic control problem is proved to be the viscosity solution to the associated SPHJB equation. A uniqueness result about viscosity solutions is also given for certain superparabolic cases, while the uniqueness of classical …
This paper is a review of results on Optimisation which are perhaps not so standard in the PDE realm. To this end, we consider the problem of deriving the PDEs associated to the optimal control of a system of either ODEs or SDEs with respect to a vector-valued cost functional. Optimisation is considered with respect to a partial ordering generated by a given cone. Since in the vector case minima may not exist, we define vectorial value functions as (Pareto) minimals of the ordering. Our main objective is the derivation of the model PDEs which turn out to be parametric families of HJB single eq…
A new stochastic control model for the long-run environmental management of rivers is mathematically and numerically analyzed, focusing on a modern sediment replenishment problem with unique nonsmooth and nonlinear properties. Rational inattention as a novel adaptive strategy to collect information and intervene against the target system is modeled using Erlangization. The system dynamics containing the river discharge following a continuous-state branching with an immigration-type process and the controlled sediment storage dynamics lead to a nonsmooth and nonlocal infinitesimal generator. Mo…
