Stochastic Differential Games and Viscosity Solutions of Hamilton-Jacobi-Bellman-Isaacs Equations
Rainer Buckdahn, Juan Li
备注：
The results were presented by Rainer Buckdahn at the "12th International Symposium on Dynamic Games and Applications" in Sophia-Antipolis (France) in June 2006; They were also reported by Juan Li at 2nd Workshop on "Stochastic Equations and Related Topics" in Jena (Germany) in July 2006 and at one seminar in the ETH of Zurich in November 2006
In this paper we study zero-sum two-player stochastic differential games with the help of theory of Backward Stochastic Differential Equations (BSDEs). At the one hand we generalize the results of the pioneer work of Fleming and Souganidis by considering cost functionals defined by controlled BSDEs and by allowing the admissible control processes to depend on events occurring before the beginning of the game (which implies that the cost functionals become random variables), on the other hand the application of BSDE methods, in particular that of the notion of stochastic "backward semigroups" introduced by Peng allows to prove a dynamic programming principle for the upper and the lower value functions of the game in a straight-forward way, without passing by additional approximations. The upper and the lower value functions are proved to be the unique viscosity solutions of the upper and the lower Hamilton-Jacobi-Bellman-Isaacs equations, respectively. For this Peng's BSDE method is translated from the framework of stochastic control theory into that of stochastic differential games.
In this paper we study a two person zero sum stochastic differential game in weak formulation. Unlike standard literature which uses strategy type of controls, the weak formulation allows us to consider the game with control against control. We shall prove the existence of game value under natural conditions. Another main feature of the paper is that we allow for non-Markovian structure, and thus the game value is a random process. We characterize the value process as the unique viscosity solution of the corresponding path dependent Bellman-Isaacs equation, a notion recently introduced by Ekre…
In this paper, we study a class of zero-sum two-player stochastic differential games with the controlled stochastic differential equations and the payoff/cost functionals of recursive type. As opposed to the pioneering work by Fleming and Souganidis [Indiana Univ. Math. J., 38 (1989), pp.~293--314] and the seminal work by Buckdahn and Li [SIAM J. Control Optim., 47 (2008), pp.~444--475], the involved coefficients may be random, going beyond the Markovian framework and leading to the random upper and lower value functions. We first prove the dynamic programming principle for the game, and then …
We investigate a two-player zero-sum stochastic differential game problem with the state process being constrained in a connected bounded closed domain, and the cost functional described by the solution of a generalized backward stochastic differential equation (GBSDE for short). We show that the value functions enjoy a (strong) dynamic programming principle, and are the unique viscosity solution of the associated Hamilton-Jacobi-Bellman-Isaacs equations with nonlinear Neumann boundary problems. To obtain the existence for viscosity solutions, we provide a new approach utilizing the representa…
We consider Hamilton Jacobi Bellman equations in an inifinite dimensional Hilbert space, with quadratic (respectively superquadratic) hamiltonian and with continuous (respectively lipschitz continuous) final conditions. This allows to study stochastic optimal control problems for suitable controlled Ornstein Uhlenbeck process with unbounded control processes.
In this paper we investigate zero-sum two-player stochastic differential games whose cost functionals are given by doubly controlled reflected backward stochastic differential equations (RBSDEs) with two barriers. For admissible controls which can depend on the whole past and so include, in particular, information occurring before the beginning of the game, the games are interpreted as games of the type "admissible strategy" against "admissible control", and the associated lower and upper value functions are studied. A priori random, they are shown to be deterministic, and it is proved that th…
For a zero-sum stochastic game which does not satisfy the Isaacs condition, we provide a value function representation for an Isaacs-type equation whose Hamiltonian lies in between the lower and upper Hamiltonians, as a convex combination of the two. For the general case (i.e. the convex combination is time and state dependent) our representation amounts to a random change of the rules of the game, to allow each player at any moment to see the other player's action or not, according to a coin toss with probabilities of heads and tails given by the convex combination appearing in the PDE. If th…
In this article, a notion of viscosity solutions is introduced for second order path-dependent Hamilton-Jacobi-Bellman (PHJB) equations associated with optimal control problems for path-dependent stochastic differential equations. We identify the value functional of optimal control problems as unique viscosity solution to the associated PHJB equations. We also show that our notion of viscosity solutions is consistent with the corresponding notion of classical solutions, and satisfies a stability property. Applications to backward stochastic Hamilton-Jacobi-Bellman equations are also given.
In this paper, a stochastic optimal control problem is investigated in which the system is governed by a stochastic functional differential equation. In the framework of functional Itô calculus, we build the dynamic programming principle and the related Path-dependent Hamilton-Jacobi-Bellman (HJB) equation. We prove that the value function is the viscosity solution of the Path-dependent HJB equation.
In this paper, we study backward doubly stochastic recursive optimal control problem where the cost function is described by the solution of a backward doubly stochastic differential equation. We give the dynamical programming principle for this kind of optimal control problem and show that the value function is the unique Sobolev weak solution for the corresponding stochastic Hamilton-Jacobi-Bellman equation.
In this paper, we study one kind of stochastic recursive optimal control problem with the obstacle constraints for the cost function where the cost function is described by the solution of one reflected backward stochastic differential equations. We will give the dynamic programming principle for this kind of optimal control problem and show that the value function is the unique viscosity solution of the obstacle problem for the corresponding Hamilton-Jacobi-Bellman equations.
We study the large time behaviour of mild solutions of HJB equations in infinite dimension by a purely probabilistic approach. For that purpose, we show that the solution of a BSDE in finite horizon $T$ taken at initial time behaves like a linear term in $T$ shifted with the solution of the associated EBSDE taken at initial time. Moreover we give an explicit speed of convergence, which seems to appear very rarely in literature.
In this paper we first investigate zero-sum two-player stochastic differential games with reflection with the help of theory of Reflected Backward Stochastic Differential Equations (RBSDEs). We will establish the dynamic programming principle for the upper and the lower value functions of this kind of stochastic differential games with reflection in a straight-forward way. Then the upper and the lower value functions are proved to be the unique viscosity solutions of the associated upper and the lower Hamilton-Jacobi-Bellman-Isaacs equations with obstacles, respectively. The method differs hea…
In this article, a notion of viscosity solutions is introduced for first order path-dependent Hamilton-Jacobi-Bellman (PHJB) equations associated with optimal control problems for path-dependent evolution equations in Hilbert space. We identify the value functional of optimal control problems as unique viscosity solution to the associated PHJB equations. We also show that our notion of viscosity solutions is consistent with the corresponding notion of classical solutions, and satisfies a stability property.
We investigate the long time behavior of weakly dissipative semilinear Hamilton-Jacobi-Bellman (HJB) equations and the turnpike property for the corresponding stochastic control problems. To this aim, we develop a probabilistic approach based on a variant of coupling by reflection adapted to the study of controlled diffusion processes. We prove existence and uniqueness of solutions for the ergodic Hamilton-Jacobi-Bellman equation and different kind of quantitative exponential convergence results at the level of the value function, of the optimal controls and of the optimal processes. Moreover,…
This paper introduces a notion of viscosity solutions for second order elliptic Hamilton-Jacobi-Bellman (HJB) equations with infinite delay associated with infinite-horizon optimal control problems for stochastic differential equations with infinite delay. We identify the value functional of optimal control problems as unique viscosity solution to associated second order elliptic HJB equation with infinite delay. We also show that our notion of viscosity solutions is consistent with the corresponding notion of classical solutions, and satisfies a stability property.
In the present work, we consider 2-person zero-sum stochastic differential games with a nonlinear pay-off functional which is defined through a backward stochastic differential equation. Our main objective is to study for such a game the problem of the existence of a value without Isaacs condition. Not surprising, this requires a suitable concept of mixed strategies which, to the authors' best knowledge, was not known in the context of stochastic differential games. For this, we consider nonanticipative strategies with a delay defined through a partition $π$ of the time interval $[0,T]$. The u…
In this article, a notion of viscosity solutions is introduced for second order path-dependent Hamilton-Jacobi-Bellman (PHJB) equations associated with optimal control problems for path-dependent stochastic evolution equations in Hilbert spaces. We identify the value functional of optimal control problems as unique viscosity solution to the associated PHJB equations. We also show that our notion of viscosity solutions is consistent with the corresponding notion of classical solutions, and satisfies a stability property.
In this paper we study the fully nonlinear stochastic Hamilton-Jacobi-Bellman (HJB) equation for the optimal stochastic control problem of stochastic differential equations with random coefficients. The notion of viscosity solution is introduced, and we prove that the value function of the optimal stochastic control problem is the maximal viscosity solution of the associated stochastic HJB equation. For the superparabolic cases when the diffusion coefficients are deterministic functions of time, states and controls, the uniqueness is addressed as well.
We aim to provide a Feynman-Kac type representation for Hamilton-Jacobi-Bellman equation, in terms of forward backward stochastic differential equation (FBSDE) with a simulatable forward process. For this purpose, we introduce a class of BSDE where the jumps component of the solution is subject to a partial nonpositive constraint. Existence and approximation of a unique minimal solution is proved by a penalization method under mild assumptions. We then show how minimal solution to this BSDE class provides a new probabilistic representation for nonlinear integro-partial differential equations (…
This is an expository note on large deviations, Hamilton-Jacobi-Bellman (HJB) equations, and the role of the Freidlin-Wentzell quasipotential in Chemical Reaction Networks (CRNs). The note was motivated by observations which identified Lyapunov functions for deterministic descriptions of CRNs by taking appropriate scaling limits of invariant distributions for the corresponding stochastic dynamics. We explain how this is a special case of a classical theory due to Freidlin and Wentzell. We also show that this Lyapunov function is a solution to the HJB partial differential equation if and only i…
We introduce a class of abstract nonlinear fractional pseudo-differential equations in Banach spaces that includes both the Mc-Kean-Vlasov-type equations describing nonlinear Markov processes and the Hamilton-Jacobi-Bellman(HJB)-Isaacs equation of stochastic control and games thus allowing for a unified analysis of these equations. This leads to an effective theory of coupled forward-backward systems (forward McKean-Vlasov evolution and backward HJB-Isaacs evolution) that are central to the modern theory of mean-field games.
Here, we study the large-time limit of viscosity solutions of the Cauchy problem for second-order Hamilton--Jacobi--Bellman equations with convex Hamiltonians in the torus. This large-time limit solves the corresponding stationary problem, sometimes called the ergodic problem. This problem, however, has multiple viscosity solutions and, thus, a key question is which of these solutions is selected by the limit. Here, we provide a representation for the viscosity solution to the Cauchy problem in terms of generalized holonomic measures. Then, we use this representation to characterize the large-…
This paper first studies super linear G-expectation. Uniqueness and existence theorem for backward stochastic differential equations (BSDEs) under super linear expectation is established to provide probabilistic interpretation for the viscosity solution of a class of Hamilton-Jacobi-Bellman equations, including the well known Black-Scholes-Barrenblett equation, arising in the uncertainty volatility model in mathematical finance. We also show that BSDEs under super linear expectation could characterize a class of stochastic control problems. A direct connection between recursive super (sub) str…
We consider the homogenization of Hamilton-Jacobi equations and degenerate Bellman equations in stationary, ergodic, unbounded environments. We prove that, as the microscopic scale tends to zero, the equation averages to a deterministic Hamilton-Jacobi equation and study some properties of the effective Hamiltonian. We discover a connection between the effective Hamiltonian and an eikonal-type equation in exterior domains. In particular, we obtain a new formula for the effective Hamiltonian. To prove the results we introduce a new strategy to obtain almost sure homogenization, completing a pro…
In this paper we investigate a kind of optimal control problem of coupled forward-backward stochastic system with jumps whose cost functional is defined through a coupled forward-backward stochastic differential equation with Brownian motion and Poisson random measure. For this end, we first study the regularity of solutions for this kind of forward-backward stochastic differential equations. We obtain that the value function is a deterministic function and satisfies the dynamic programming principle for this kind of optimal control problem. Moreover, we prove that the value functions is a vis…
We study the regularity properties of integro-partial differential equations of Hamilton-Jocobi-Bellman type with terminal condition, which can be interpreted through a stochastic control system, composed of a forward and a backward stochastic differential equation, both driven by a Brownian motion and a compensated Poisson random measure. More precisely, we prove that, under appropriate assumptions, the viscosity solution of such equations is jointly Lipschitz and jointly semiconcave in $(t,x)\inΔ\times\R^d$, for all compact time intervals $Δ$ excluding the terminal time. Our approach is base…
In this paper we study zero-sum two-player stochastic differential games with jumps with the help of theory of Backward Stochastic Differential Equations (BSDEs). We generalize the results of Fleming and Souganidis [10] and those by Biswas [3] by considering a controlled stochastic system driven by a d-dimensional Brownian motion and a Poisson random measure and by associating nonlinear cost functionals defined by controlled BSDEs. Moreover, unlike the both papers cited above we allow the admissible control processes of both players to depend on all events occurring before the beginning of the…
In this paper, we propose and study the stochastic path-dependent Hamilton-Jacobi-Bellman (SPHJB) equation that arises naturally from the optimal stochastic control problem of stochastic differential equations with path-dependence and measurable randomness. Both the notions of viscosity solution and classical solution are proposed, and the value function of the optimal stochastic control problem is proved to be the viscosity solution to the associated SPHJB equation. A uniqueness result about viscosity solutions is also given for certain superparabolic cases, while the uniqueness of classical …
In the present paper we investigate the problem of the existence of a value for differential games without Isaacs condition. For this we introduce a suitable concept of mixed strategies along a partition of the time interval, which are associated with classical nonanticipative strategies (with delay). Imposing on the underlying controls for both players a conditional independence property, we obtain the existence of the value in mixed strategies as the limit of the lower as well as of the upper value functions along a sequence of partitions which mesh tends to zero. Moreover, we characterize t…
This paper is concerned with stochastic differential games (SDGs) defined through fully coupled forward-backward stochastic differential equations (FBSDEs) which are governed by Brownian motion and Poisson random measure. For SDGs, the upper and the lower value functions are defined by the controlled fully coupled FBSDEs with jumps. Using a new transformation introduced in [6], we prove that the upper and the lower value functions are deterministic. Then, after establishing the dynamic programming principle for the upper and the lower value functions of this SDGs, we prove that the upper and t…
This paper is a review of results on Optimisation which are perhaps not so standard in the PDE realm. To this end, we consider the problem of deriving the PDEs associated to the optimal control of a system of either ODEs or SDEs with respect to a vector-valued cost functional. Optimisation is considered with respect to a partial ordering generated by a given cone. Since in the vector case minima may not exist, we define vectorial value functions as (Pareto) minimals of the ordering. Our main objective is the derivation of the model PDEs which turn out to be parametric families of HJB single eq…
Universität Potsdam（波茨坦大学）；Freie Universität Berlin（柏林自由大学）；Brandenburgische Technische Universität Cottbus-Senftenberg（勃兰登堡工业大学科特布斯-森夫滕贝格）
Optimal control of diffusion processes is intimately connected to the problem of solving certain Hamilton-Jacobi-Bellman equations. Building on recent machine learning inspired approaches towards high-dimensional PDEs, we investigate the potential of $\textit{iterative diffusion optimisation}$ techniques, in particular considering applications in importance sampling and rare event simulation, and focusing on problems without diffusion control, with linearly controlled drift and running costs that depend quadratically on the control. More generally, our methods apply to nonlinear parabolic PDEs…
