This paper presents a novel approach to numerically solve stochastic differential games for nonlinear systems. The proposed approach relies on the nonlinear Feynman-Kac theorem that establishes a connection between parabolic deterministic partial differential equations and forward-backward stochastic differential equations. Using this theorem the Hamilton-Jacobi-Isaacs partial differential equation associated with differential games is represented by a system of forward-backward stochastic differential equations. Numerical solution of the aforementioned system of stochastic differential equations is performed using importance sampling and a Long-Short Term Memory recurrent neural network, which is trained in an offline fashion. The resulting algorithm is tested on two example systems in simulation and compared against the standard risk neutral stochastic optimal control formulations.
We consider a mean field game with common noise in which the diffusion coefficients may be controlled. We prove existence of a weak relaxed solution under some continuity conditions on the coefficients. We then show that, when there is no common noise, the solution of this mean field game is characterized by a McKean-Vlasov type second order backward SDE.
We study optimal stochastic control problems of general coupled systems of forward-backward stochastic differential equations with jumps. By means of the Itô-Ventzell formula the system is transformed to a controlled backward stochastic partial differential equation (BSPDE) with jumps. Using a comparison principle for such BSPDEs we obtain a general stochastic Hamilton-Jacobi- Bellman (HJB) equation for such control problems. In the classical Markovian case with optimal control of jump diffusions, the equation reduces to the classical HJB equation. The results are applied to study risk minimiz…
This work deals with backward stochastic differential equation (BSDE) with random marked jumps, and their applications to default risk. We show that these BSDEs are linked with Brownian BSDEs through the decomposition of processes with respect to the progressive enlargement of filtrations. We show that the equations have solutions if the associated Brownian BSDEs have solutions. We also provide a uniqueness theorem for BSDEs with jumps by giving a comparison theorem based on the comparison for Brownian BSDEs. We give in particular some results for quadratic BDSEs. As applications, we study the…
We propose a new method for the numerical solution of the forward-backward stochastic differential equations (FBSDE) appearing in the Feynman-Kac representation of the value function in stochastic optimal control problems. Using Girsanov's change of probability measures, it is demonstrated how a McKean-Markov branched sampling method can be utilized for the forward integration pass, as long as the controlled drift term is appropriately compensated in the backward integration pass. Subsequently, a numerical approximation of the value function is proposed by solving a series of function approxim…
We address a general optimal switching problem over finite horizon for a stochastic system described by a differential equation driven by Brownian motion. The main novelty is the fact that we allow for infinitely many modes (or regimes, i.e. the possible values of the piecewise-constant control process). We allow all the given coefficients in the model to be path-dependent, that is, their value at any time depends on the past trajectory of the controlled system. The main aim is to introduce a suitable (scalar) backward stochastic differential equation (BSDE), with a constraint on the martingal…
Two novel numerical estimators are proposed for solving forward-backward stochastic differential equations (FBSDEs) appearing in the Feynman-Kac representation of the value function in stochastic optimal control problems. In contrast to the current numerical approaches which are based on the discretization of the continuous-time FBSDE, we propose a converse approach, namely, we obtain a discrete-time approximation of the on-policy value function, and then we derive a discrete-time estimator that resembles the continuous-time counterpart. The proposed approach allows for the construction of hig…
We study an optimal control problem on infinite horizon for a controlled stochastic differential equation driven by Brownian motion, with a discounted reward functional. The equation may have memory or delay effects in the coefficients, both with respect to state and control, and the noise can be degenerate. We prove that the value, i.e. the supremum of the reward functional over all admissible controls, can be represented by the solution of an associated backward stochastic differential equation (BSDE) driven by the Brownian motion and an auxiliary independent Poisson process and having a sig…
This paper is concerned with a backward-forward stochastic differential equation (BFSDE) system, in which a large number of negligible agents are coupled in their dynamics via state average. Here some BSDE is introduced as the dynamics of major player, while dynamics of minor players are described by SDEs. Some auxiliary mean-field SDEs (MFSDEs) and a $3\times2$ mixed forward-backward stochastic differential equation (FBSDE) system are considered and analyzed instead of involving the fixed-point analysis as in other mean-field games. We also derive the decentralized strategies which are shown …
We study a novel general class of multidimensional type-I backward stochastic Volterra integral equations. Toward this goal, we introduce an infinite dimensional system of standard backward SDEs and establish its well-posedness, and we show that it is equivalent to that of a type-I backward stochastic Volterra integral equation. We also establish a representation formula in terms of non-linear semilinear partial differential equation of Hamilton-Jacobi-Bellman type. As an application, we consider the study of time-inconsistent stochastic control from a game-theoretic point of view. We show the…
University of Basel（巴塞尔大学）；ELLIS Institute Tübingen（埃利斯研究所蒂宾根分所）；MPI for Intelligent Systems Tübingen（蒂宾根智能系统马克斯·普朗克研究所）；AI Center（人工智能中心）；Yahoo! Research（雅虎研究院）；University of Oslo（奥斯陆大学）
Devising optimal interventions for diffusive systems often requires the solution of the Hamilton-Jacobi-Bellman (HJB) equation, a nonlinear backward partial differential equation (PDE), that is, in general, nontrivial to solve. Existing control methods either tackle the HJB directly with grid-based PDE solvers, or resort to iterative stochastic path sampling to obtain the necessary controls. Here, we present a framework that interpolates between these two approaches. By reformulating the optimal interventions in terms of logarithmic gradients ( scores ) of two forward probability flows, and by…
Convergence analysis of controlled particle systems arising in deep learning: from finite to infinite sample size
School of Mathematical Sciences, Dalian University of Technology（大连理工大学数学科学学院）；Department of Mathematical Sciences, University of Durham（杜伦大学数学科学学院）；Department of Mathematics, City University of Hong Kong（香港城市大学数学系）；Department of Mathematics and Risk Management Institute, National University of Singapore（新加坡国立大学数学系与风险管理研究所）
We study four systems and their interactions. First, we formulate a unified system of coupled forward-backward stochastic partial differential equations (FB-SPDEs) with Levy jumps, whose drift, diffusion, and jump coefficients may involve partial differential operators. A solution to the FB-SPDEs is defined by a 4-tuple general dimensional random vector-field process evolving in time together with position parameters over a domain (e.g., a hyperbox or a manifold). Under an infinite sequence of generalized local linear growth and Lipschitz conditions, the well-posedness of an adapted 4-tuple st…
This paper studies a new class of dynamic optimization problems of large-population (LP) system which consists of a large number of negligible and coupled agents. The most significant feature in our setup is the dynamics of individual agents follow the forward-backward stochastic differential equations (FBSDEs) in which the forward and backward states are coupled at the terminal time. This current paper is hence different to most existing large-population literature where the individual states are typically modeled by the SDEs including the forward state only. The associated mean-field linear-…
In this paper, we study the optimal control of a discrete-time stochastic differential equation (SDE) of mean-field type, where the coefficients can depend on both a function of the law and the state of the process. We establish a new version of the maximum principle for discrete-time stochastic optimal control problems. Moreover, the cost functional is also of the mean-field type. This maximum principle differs from the classical principle since we introduce new discrete-time backward (matrix) stochastic equations. Based on the discrete-time backward stochastic equations where the adjoint equ…
This paper studies open-loop equilibriums for a general class of time-inconsistent stochastic control problems under jump-diffusion SDEs with deterministic coefficients. Inspired by the idea of Four-Step-Scheme for forward-backward stochastic differential equations with jumps (FBSDEJs, for short), we derive two systems of integro-partial differential equations (IPDEs, for short). Then, we rigorously prove a verification theorem which provides a sufficient condition for open-loop equilibrium strategies. As an illustration of the general theory, we discuss a mean-variance portfolio selection pro…
