We study the optimal control of path-dependent McKean-Vlasov equations valued in Hilbert spaces motivated by non Markovian mean-field models driven by stochastic PDEs. We first establish the well-posedness of the state equation, and then we prove the dynamic programming principle (DPP) in such a general framework. The crucial law invariance property of the value function V is rigorously obtained, which means that V can be viewed as a function on the Wasserstein space of probability measures on the set of continuous functions valued in Hilbert space. We then define a notion of pathwise measure derivative, which extends the Wasserstein derivative due to Lions [41], and prove a related functional It{ô} formula in the spirit of Dupire [24] and Wu and Zhang [51]. The Master Bellman equation is derived from the DPP by means of a suitable notion of viscosity solution. We provide different formulations and simplifications of such a Bellman equation notably in the special case when there is no dependence on the law of the control.
We consider a mean field game with common noise in which the diffusion coefficients may be controlled. We prove existence of a weak relaxed solution under some continuity conditions on the coefficients. We then show that, when there is no common noise, the solution of this mean field game is characterized by a McKean-Vlasov type second order backward SDE.
This paper rigorously connects the problem of optimal control of McKean-Vlasov dynamics with large systems of interacting controlled state processes. Precisely, the empirical distributions of near-optimal control-state pairs for the $n$-state systems, as $n$ tends to infinity, admit limit points in distribution (if the objective functions are suitably coercive), and every such limit is supported on the set of optimal control-state pairs for the McKean-Vlasov problem. Conversely, any distribution on the set of optimal control-state pairs for the McKean-Vlasov problem can be realized as a limit …
In this paper we study the sensitivity of nonlinear stochastic differential equations of McKean-Vlasov type generated by stable-like processes. By using the method of stochastic characteristics, we transfer these equations to the non-stochastic equations with random coefficients thus making it possible to use the results obtained for nonlinear PDE of McKean-Vlasov type generated by stable-like processes in the previous works. The motivation for studying sensitivity of nonlinear McKean-Vlasov SPDEs arises naturally from the analysis of the mean-field games with common noise.
The purpose of this paper is to study optimal control of conditional McKean-Vlasov (mean-field) stochastic differential equations with jumps (conditional McKean-Vlasov jump diffusions, for short). To this end, we first prove a stochastic Fokker-Planck equation for the conditional law of the solution of such equations. Combining this equation with the original state equation, we obtain a Markovian system for the state and its conditional law. Furthermore, we apply this to formulate an Hamilton-Jacobi-Bellman (HJB) equation for the optimal control of conditional McKean-Vlasov jump diffusions. Th…
We consider the stochastic optimal control problem of nonlinear mean-field systems in discrete time. We reformulate the problem into a deterministic control problem with marginal distribution as controlled state variable, and prove that dynamic programming principle holds in its general form. We apply our method for solving explicitly the mean-variance portfolio selection and the multivariate linear-quadratic McKean-Vlasov control problem.
We study a class of non linear integro-differential equations on the Wasserstein space related to the optimal control of McKean--Vlasov jump-diffusions. We develop an intrinsic notion of viscosity solutions that does not rely on the lifting to an Hilbert space and prove a comparison theorem for these solutions. We also show that the value function is the unique viscosity solution.
In this paper, we show existence and uniqueness of solutions of the infinite horizon McKean-Vlasov FBSDEs using two different methods, which lead to two different sets of assumptions. We use these results to solve the infinite horizon mean field type control problems and mean field games.
We study the problem of optimal stopping of conditional McKean-Vlasov (mean-field) stochastic differential equations with jumps (conditional McKean-Vlasov jump diffusions, for short). We obtain sufficient variational inequalities for a function to be the value function of such a problem and for a stopping time to be optimal. To achieve this, we combine the state equation for the conditional McKean-Vlasov equation with the associated stochastic Fokker-Planck equation for the conditional law of the solution of the state. This gives us a Markovian system which can be handled by using a version of…
We study the McKean-Vlasov optimal control problem with common noise in various formulations, namely the strong and weak formulation, as well as the Markovian and non-Markovian formulations, and allowing for the law of the control process to appear in the state dynamics. By interpreting the controls as probability measures on an appropriate canonical space with two filtrations, we then develop the classical measurable selection, conditioning and concatenation arguments in this new context, and establish the dynamic programming principle under general conditions.
We study zero-sum stochastic differential games where the state dynamics of the two players is governed by a generalized McKean-Vlasov (or mean-field) stochastic differential equation in which the distribution of both state and controls of each player appears in the drift and diffusion coefficients, as well as in the running and terminal payoff functions. We prove the dynamic programming principle (DPP) in this general setting, which also includes the control case with only one player, where it is the first time that DPP is proved for open-loop controls. We also show that the upper and lower v…
In this paper, we generalise Pontryagin's stochastic maximum principle to controlled McKean-Vlasov equations with anticipating law. The associated new type of delayed backward equations with implicit terminal condition is studied.
The paper deals with a class of time-inconsistent control problems for McKean-Vlasov dynamics. By solving a backward time-inconsistent Hamilton-Jacobi-Bellman (HJB for short) equation coupled with a forward distribution-dependent stochastic differential equation, we investigate the existence and uniqueness of a closed-loop equilibrium for such time-inconsistent distribution-dependent control problem. Moreover, a special case of semi-linear McKean-Vlasov dynamics with a quadratic-type cost functional is considered due to its special structure.
The purpose of this paper is to establish a stochastic differential equation for the Donsker delta measure of the solution of a McKean-Vlasov (mean-field) stochastic differential equation. If the Donsker delta measure is absolutely continuous with respect to Lebesgue measure, then its Radon-Nikodym derivative is called the Donsker delta function. In that case it can be proved that the local time of such a process is simply the integral with respect to time of the Donsker delta function. Therefore we also get an equation for the local time of such a process. For some particular McKean-Vlasov pr…
We consider conditional McKean-Vlasov stochastic differential equations (SDEs), such as the ones arising in the large-system limit of mean field games and particle systems with mean field interactions when common noise is present. The conditional time-marginals of the solutions to these SDEs satisfy non-linear stochastic partial differential equations (SPDEs) of the second order, whereas the laws of the conditional time-marginals follow Fokker-Planck equations on the space of probability measures. We prove two superposition principles: The first establishes that any solution of the SPDE can be…
We establish an algebraic rate of convergence in the large number of players limit of the value functions of N-particle stochastic control problems towards the value function of the corresponding McKean-Vlasov problem also known as mean field control. The rate is obtained in the presence of both idiosyncratic and common noises and in a setting where the value function for the McKean-Vlasov problem need not be smooth. Our approach relies crucially on uniform in N Lipschitz and semi-concavity estimates for the N-particle value functions as well as a certain concentration inequality.
We propose several algorithms to solve McKean-Vlasov Forward Backward Stochastic Differential Equations. Our schemes rely on the approximating power of neural networks to estimate the solution or its gradient through minimization problems. As a consequence, we obtain methods able to tackle both mean field games and mean field control problems in moderate dimension. We analyze the numerical behavior of our algorithms on several examples including non linear quadratic models.
