In this article, we concern a kind of partially observed non-zero sum stochastic differential game based on forward and backward stochastic differential equations (FBSDEs). It is required that each player has his own observation equation, and the corresponding open-loop Nash equilibrium control is required to adapted to the filtration that the observation process generated. To find this open-loop Nash equilibrium point, we prove the maximum principle as a necessary condition of the existence of this point, and give a verification theorem as a sufficient condition to verify it is the real open-loop Nash equilibrium point. Combined this with reality, a financial investment problem is raised. We can obtain the explicit observable investment strategy by using stochastic filtering theory and the results above.
The purpose of this paper is to explore the necessary conditions for optimality of mean-field forward-backward delay control systems. A new estimate is proved, which is a powerfultool to deal with the optimal control problems of mean-field type with delay. Different from the classical situation, in our case the first-order adjoint system is an anticipated mean-field backward stochastic differential equation, and the second-order adjoint system is a system of matrix-valued process, not mean-field type.With the help of two adjoint systems, the second-order expansion of the variation of the state…
This paper is concerned with a discounted optimal control problem of partially observed forward-backward stochastic systems with jumps on infinite horizon. The control domain is convex and a kind of infinite horizon observation equation is introduced. The uniquely solvability of infinite horizon forward (backward) stochastic differential equation with jumps is obtained and more extended analysis, especially for the backward case, is made. Some new estimates are first given and proved for the critical variational inequality. Then an ergodic maximum principle is obtained by introducing some infi…
IIn this paper, we study a partially observed progressive optimal control problem of forward-backward stochastic differential equations with random jumps, where the control domain is not necessarily convex, and the control variable enter into all the coefficients. In our model, the observation equation is not only driven by a Brownian motion but also a Poisson random measure, which also have correlated noises with the state equation. For preparation, we first derive the existence and uniqueness of the solutions to the fully coupled forward-backward stochastic system with random jumps and its e…
Memory-limited partially observable stochastic control (ML-POSC) is the stochastic optimal control problem under incomplete information and memory limitation. In order to obtain the optimal control function of ML-POSC, a system of the forward Fokker-Planck (FP) equation and the backward Hamilton-Jacobi-Bellman (HJB) equation needs to be solved. In this work, we firstly show that the system of HJB-FP equations can be interpreted via the Pontryagin's minimum principle on the probability density function space. Based on this interpretation, we then propose the forward-backward sweep method (FBSM)…
This article is concerned with an optimal control problem derived by mean-field forward-backward stochastic differential equation with noisy observation, where the drift coefficients of the state equation and the observation equation are linear with respect to the state and its expectation. The control problem is different from the existing literature on optimal control for mean-field stochastic systems, and has more applications in mathematical finance, e.g., asset-liability management problem with recursive utility, systematic risk model. Using a backward separation method with a decompositi…
We present semi-decentralized and distributed algorithms, designed via a preconditioned forward-backward operator splitting, for solving large-scale, decomposable semidefinite programs (SDPs). We exploit a chordal aggregate sparsity pattern assumption on the original SDP to obtain a set of mutually coupled SDPs defined on positive semidefinite (PSD) cones of reduced dimensions. We show that the proposed algorithms converge to a solution of the original SDP via iterations of reasonable computational cost. Finally, we compare the performances of the two proposed algorithms with respect to others…
Fractional Backward Stochastic Partial Differential Equations with Applications to Stochastic Optimal Control of Partially Observed Systems driven by Lévy Processes
In this paper we study the mean-field backward stochastic differential equations (mean-field bsde) of the form dY(t) =-f(t,Y(t),Z(t),K(t, . ),E[φ(Y(t),Z(t),K(t,.))])dt+Z(t)dB(t) +\int_{R_{0}}K(t,ζ)\tilde{N}(dt,dζ), where B is a Brownian motion, \tilde{N} is the compensated Poisson random measure. Under some mild conditions, we prove the existence and uniqueness of the solution triplet (Y,Z,K). It is commonly believed that there is no comparison theorem for general mean-field bsde. However, we prove a comparison theorem for a subclass of these equations. When the mean-field bsde is linear, we g…
A linear-quadratic partially observed Stackelberg stochastic differential game with multiple followers and its application to multi-agent formation control
East China University of Science and Technology（华东理工大学）；University of Alberta（阿尔伯塔大学）；Shandong University（山东大学）；Fudan University（复旦大学）
In this paper, we study the maximum principle for stochastic optimal control problems of forward-backward stochastic difference systems (FBSΔSs). Two types of FBSΔSs are investigated. The first one is described by a partially coupled forward-backward stochastic difference equation (FBSΔE) and the second one is described by a fully coupled FBSΔE. By adopting an appropriate representation of the product rule and an appropriate formulation of the backward stochastic difference equation (BSΔE), we deduce the adjoint difference equation. Finally, the maximum principle for this optimal control probl…
We discuss an open-loop backward Stackelberg differential game involving single leader and single follower. Unlike most Stackelberg game literature, the state to be controlled is characterized by a backward stochastic differential equation (BSDE) for which the terminal- instead initial-condition is specified as a priori; the decisions of leader consist of a static terminal-perturbation and a dynamic linear-quadratic control. In addition, the terminal control is subject to (convex-closed) pointwise and (affine) expectation constraints. Both constraints are arising from real applications such as…
We consider an energy storage problem involving a wind farm with a forecasted power output, a stochastic load, an energy storage device, and a connection to the larger power grid with stochastic prices. Electricity prices and wind power forecast errors are modeled using a novel hidden semi-Markov model that accurately replicates not just the distribution of the errors, but also crossing times, capturing the amount of time each process stays above or below some benchmark such as the forecast. This is an important property of stochastic processes involved in storage problems. We show that we ach…
This paper first studies super linear G-expectation. Uniqueness and existence theorem for backward stochastic differential equations (BSDEs) under super linear expectation is established to provide probabilistic interpretation for the viscosity solution of a class of Hamilton-Jacobi-Bellman equations, including the well known Black-Scholes-Barrenblett equation, arising in the uncertainty volatility model in mathematical finance. We also show that BSDEs under super linear expectation could characterize a class of stochastic control problems. A direct connection between recursive super (sub) str…
This paper is concerned with a linear-quadratic partially observed Stackelberg stochastic differential game with correlated state and observation noises, where the diffusion coefficient does not contain the control variable and the control set is not necessarily convex. Both the leader and the follower have their own observation equations, and the information filtration available to the leader is contained in that to the follower. By spike variational, state decomposition and backward separation techniques, necessary and sufficient conditions of the Stackelberg equilibrium points are derived. …
This paper deals with partially-observed optimal control problems for the state governed by stochastic differential equation with delay. We develop a stochastic maximum principle for this kind of optimal control problems using a variational method and a filtering technique. Also, we establish a sufficient condition without assumption of the concavity. Two examples shed light on the theoretical results are established in the paper. In particular, in the example of an optimal investment problem with delay, its numerical simulation shows the effect of delay via a discretization technique for forw…
We study a coupled system of controlled stochastic differential equations (SDEs) driven by a Brownian motion and a compensated Poisson random measure, consisting of a forward SDE in the unknown process $X(t)$ and a \emph{predictive mean-field} backward SDE (BSDE) in the unknowns $Y(t), Z(t), K(t,\cdot)$. The driver of the BSDE at time $t$ may depend not just upon the unknown processes $Y(t), Z(t), K(t,\cdot)$, but also on the predicted future value $Y(t+δ)$, defined by the conditional expectation $A(t):= E[Y(t+δ) | \mathcal{F}_t]$. \\ We give a sufficient and a necessary maximum principle for …
Linear-quadratic mixed Stackelberg-zero-sum game for mean-field regime switching system
Shandong University of Finance and Economics（山东财经大学）；Dalian University of Technology（大连理工大学）；The Hong Kong Polytechnic University（香港理工大学）；Imperial College（帝国理工学院）
In this paper, we study the maximum principle for stochastic optimal control problems of forward-backward stochastic difference systems (FBSΔSs) where the uncertainty is modeled by a discrete time, finite state process, rather than white noises. Two types of FBSΔSs are investigated. The first one is described by a partially coupled forward-backward stochastic difference equation (FBSΔE) and the second one is described by a fully coupled FBSΔE. By adopting an appropriate representation of the product rule and an appropriate formulation of the backward stochastic difference equation (BSΔE), we d…
This paper is concerned with the stochastic Hamilton-Jacobi-Bellman equation with controlled leading coefficients, which is a type of fully nonlinear backward stochastic partial differential equation (BSPDE for short). In order to formulate the weak solution for such kind of BSPDEs, the classical potential theory is generalized in the backward stochastic framework. The existence and uniqueness of the weak solution is proved, and for the partially non-Markovian case, we obtain the associated gradient estimate. As a byproduct, the existence and uniqueness of solution for a class of degenerate re…
Decentralized partially observable Markov decision process (DEC-POMDP) models sequential decision making problems by a team of agents. Since the planning of DEC-POMDP can be interpreted as the maximum likelihood estimation for the latent variable model, DEC-POMDP can be solved by the EM algorithm. However, in EM for DEC-POMDP, the forward--backward algorithm needs to be calculated up to the infinite horizon, which impairs the computational efficiency. In this paper, we propose the Bellman EM algorithm (BEM) and the modified Bellman EM algorithm (MBEM) by introducing the forward and backward Be…
This paper investigates a class of unified stochastic linear quadratic Gaussian (LQG) social optima problems involving a large number of weakly-coupled interactive agents under a {generalized} setting. For each individual agent, the control and state process enters both diffusion and drift terms in its linear dynamics, and the control weight might be \emph{indefinite} in cost functional. This setup is {innovative and has great theoretical and realistic significance} as its applications in mathematical finance {(e.g., portfolio selection in mean-variation model)}. Using some \emph{fully-coupled…
This paper is concerned with a partially observed hybrid optimal control problem, where continuous dynamics and discrete events coexist and in particular, the continuous dynamics can be observed while the discrete events, described by a Markov chain, is not directly available. Such kind of problem is first considered in the literature and has wide applications in finance, management, engineering, and so on. There are three major contributions made in this paper: First, we develop a novel non-linear filtering method to convert the partially observed problem into a completely observed one. Our m…
We revisit the problem of estimating the parameters of a partially observed diffusion process, consisting of a hidden state process and an observed process, with a continuous time parameter. The estimation is to be done online, i.e. the parameter estimate should be updated recursively based on the observation filtration. We provide a theoretical analysis of the stochastic gradient ascent algorithm on the incomplete-data log-likelihood. The convergence of the algorithm is proved under suitable conditions regarding the ergodicity of the process consisting of state, filter, and tangent filter. Ad…
Particle Methods with Deep Learning for Stochastic Control under Partial Observation
Shanghai Center for Data Science（上海数据科学中心）；NYU-ECNU Institute of Mathematical Sciences, NYU Shanghai（纽约大学华东师范大学数学科学联合研究所）；Department of Mathematics, The Chinese University of Hong Kong（香港中文大学数学系）
One of the fundamental issues in Control Theory is to design feedback controls. It is well-known that, the purpose of introducing Riccati equations in the deterministic case is to provide the desired feedback controls for linear quadratic control problems. To date, the same problem in the stochastic setting is only partially well-understood. In this paper, we establish the equivalence between the existence of optimal feedback controls for the stochastic linear quadratic control problems with random coefficients and the solvability of the corresponding backward stochastic Riccati equations in a…
The Coca-Cola Company（可口可乐公司）；Georgia Institute of Technology（佐治亚理工学院）
We consider a class of sequential decision-making problems under uncertainty that can encompass various types of supervised learning concepts. These problems have a completely observed state process and a partially observed modulation process, where the state process is affected by the modulation process only through an observation process, the observation process only observes the modulation process, and the modulation process is exogenous to control. We model this broad class of problems as a partially observed Markov decision process (POMDP). The belief function for the modulation process i…
In this paper we focus on a general type of mean-field stochastic control problem with partial observation, in which the coefficients depend in a non-linear way not only on the state process $X_t$ and its control $u_t$ but also on the conditional law $E[X_t|\mathcal{F}_t^Y]$ of the state process conditioned with respect to the past of observation process $Y$. We first deduce the well-posedness of the controlled system by showing weak existence and uniqueness in law. Neither supposing convexity of the control state space nor differentiability of the coefficients with respect to the control vari…
Princeton University（普林斯顿大学）；University of California, Santa Barbara（加州大学圣巴巴拉分校）
Stochastic differential games have been used extensively to model agents' competitions in Finance, for instance, in P2P lending platforms from the Fintech industry, the banking system for systemic risk, and insurance markets. The recently proposed machine learning algorithm, deep fictitious play, provides a novel efficient tool for finding Markovian Nash equilibrium of large $N$-player asymmetric stochastic differential games [J. Han and R. Hu, Mathematical and Scientific Machine Learning Conference, pages 221-245, PMLR, 2020]. By incorporating the idea of fictitious play, the algorithm decoup…
