In this paper, we study a time-inconsistent stochastic optimal control problem with a recursive cost functional by a multi-person hierarchical differential game approach. An equilibrium strategy of this problem is constructed and a corresponding equilibrium Hamilton-Jacobi-Bellman (HJB, for short) equation is established to characterize the associated equilibrium value function. Moreover, a well-posedness result of the equilibrium HJB equation is established under certain conditions.
In this paper, we study extended backward stochastic Volterra integral equations (EBSVIEs, for short). We establish the well-posedness under weaker assumptions than the literature, and prove a new kind of regularity property for the solutions. As an application, we investigate, in the open-loop framework, a time-inconsistent stochastic recursive control problem where the cost functional is defined by the solution to a backward stochastic Volterra integral equation (BSVIE, for short). We show that the corresponding adjoint equations become EBSVIEs, and provide a necessary and sufficient conditi…
We study a stochastic recursive optimal control problem in which the cost functional is described by the solution of a backward stochastic differential equation driven by G-Brownian motion. Some of the economic and financial optimization problems with volatility ambiguity can be formulated as such problems. Different from the classical variational approach, we establish the maximum principle by the linearization and weak convergence methods.
This paper studies open-loop equilibriums for a general class of time-inconsistent stochastic control problems under jump-diffusion SDEs with deterministic coefficients. Inspired by the idea of Four-Step-Scheme for forward-backward stochastic differential equations with jumps (FBSDEJs, for short), we derive two systems of integro-partial differential equations (IPDEs, for short). Then, we rigorously prove a verification theorem which provides a sufficient condition for open-loop equilibrium strategies. As an illustration of the general theory, we discuss a mean-variance portfolio selection pro…
This paper deals with a class of time inconsistent stochastic linear quadratic (SLQ) optimal control problems in Markovian framework. Three notions, i.e., closed-loop equilibrium controls/strategies, open-loop equilibrium controls and their closed-loop representations, are characterized in unified manners. These results indicate clearer and deeper distinctions among these notions. For example, in particular time consistent setting, the open-loop equilibrium controls are fully characterized by first-order, second-order necessary optimality conditions, and become needlessly optimal, while the cl…
This paper aims to explore the relationship between maximum principle and dynamic programming principle for stochastic recursive control problem with random coefficients. Under certain regular conditions for the coefficients, the relationship between the Hamilton system with random coefficients and stochastic Hamilton-Jacobi-Bellman equation is obtained. It is very different from the deterministic coefficients case since stochastic Hamilton-Jacobi-Bellman equation is a backward stochastic partial differential equation with solution being a pair of random fields rather than a deterministic func…
In this paper, we study a stochastic recursive optimal control problem in which the objective functional is described by the solution of a backward stochastic differential equation driven by G-Brownian motion. Under standard assumptions, we establish the dynamic programming principle and the related Hamilton-Jacobi-Bellman (HJB) equation in the framework of G-expectation. Finally, we show that the value function is the viscosity solution of the obtained HJB equation.
In this paper, we study a stochastic recursive optimal control problem in which the cost functional is described by the solution of a backward stochastic differential equation driven by G-Brownian motion. Under standard assumptions, we establish the dynamic programming principle and the related fully nonlinear HJB equation in the framework of G-expectation. Finally, we show that the value function is the viscosity solution of the obtained HJB equation.
This paper is concerned with the stochastic recursive optimal control problem with mixed delay. The connection between Pontryagin's maximum principle and Bellman's dynamic programming principle is discussed. Without containing any derivatives of the value function, relations among the adjoint processes and the value function are investigated by employing the notions of super- and sub-jets introduced in defining the viscosity solutions. Stochastic verification theorem is also given to verify whether a given admissible control is really optimal.
We propose a novel stochastic optimization algorithm called STOchastic Recursive Momentum for Compositional (STORM-Compositional) optimization that minimizes the composition of expectations of two stochastic functions, the latter being an optimization problem arising in various important machine learning applications. By introducing the momentum term in the compositional gradient updates, STORM-Compositional operates the stochastic recursive variance-reduced compositional gradients in an exponential-moving average way. This leads to an $O(\varepsilon^{-3})$ complexity upper bound for STORM-Com…
In this paper, we establish a general stochastic maximum principle for optimal control for systems described by a continuous-time Markov regime-switching stochastic recursive utilities model. The control domain is postulated not to be convex, and the diffusion terms depend on control variables. To this end, we first study a kind of classical forward stochastic optimal control problems. Afterwards, based on previous results, we introduce two groups of new first and second-order adjoint equations. The corresponding variational equations for forward-backward stochastic differential equations are …
In this paper, we study the optimal singular controls for stochastic recursive systems, in which the control has two components: the regular control, and the singular control. Under certain assumptions, we establish the dynamic programming principle for this kind of optimal singular controls problem, and prove that the value function is a unique viscosity solution of the corresponding Hamilton-Jacobi-Bellman inequality, in a given class of bounded and continuous functions. At last, an example is given for illustration.
This paper deals with a nonsmooth version of the connection between the maximum principle and dynamic programming principle, for the stochastic recursive control problem when the control domain is convex. By employing the notions of sub- and super-jets, the set inclusions are derived among the value function and the adjoint processes. The general case for non-convex control domain is open.
In this paper, we study the relationship between maximum principle (MP) and dynamic programming principle (DPP) for stochastic recursive optimal control problem driven by $G$-Brownian motion. Under the smooth assumption for the value function, we obtain the connection between MP and DPP under a reference probability $P_{t,x}^{\ast}$. Within the framework of viscosity solution, we establish the relation between the first-order super-jet, sub-jet of the value function and the solution to the adjoint equation respectively.
An optimal control problem is considered for a stochastic differential equation containing a state-dependent regime switching, with a recursive cost functional. Due to the non-exponential discounting in the cost functional, the problem is time-inconsistent in general. Therefore, instead of finding a global optimal control (which is not possible), we look for a time-consistent (approximately) locally optimal equilibrium strategy. Such a strategy can be represented through the solution to a system of partial differential equations, called an equilibrium Hamilton-Jacob-Bellman (HJB, for short) eq…
In this paper, a class of time inconsistent linear quadratic optimal control problems of mean-field stochastic differential equations (SDEs) is considered under Markovian framework. Open-loop equilibrium controls and their particular closed-loop representations are introduced and characterized via variational ideas. Several interesting features are revealed and a system of coupled Riccati equations is derived. In contrast with the analogue optimal control problems of SDEs, the mean-field terms in state equation, which is another reason of time inconsistency, prompts us to define above two noti…
In this paper, we study a class of stochastic time-inconsistent linear-quadratic (LQ) control problems with control input constraints. These problems are investigated within the more general framework associated with random coefficients. This paper aims to further develop a new methodology, which fundamentally differs from those in the standard control (without constraints) theory in the literature, to cope with the mathematical difficulties raised due to the presence of input constraints. We first prove that the existence of an equilibrium solution is equivalent to the existence of a solution…
This paper focuses on linear-quadratic (LQ for short) mean-field games described by forward-backward stochastic differential equations (FBSDEs for short), in which the individual control region is postulated to be convex. The decentralized strategies and consistency condition are represented by a kind of coupled mean-field FBSDEs with projection operators. The well-posedness of consistency condition system is obtained using the monotonicity condition method. The $ε$-Nash equilibrium property is discussed as well.
In this work we study the stochastic recursive control problem, in which the aggregator (or called generator) of the backward stochastic differential equation describing the running cost is continuous but not necessarily Lipschitz with respect to the first unknown variable and the control, and monotonic with respect to the first unknown variable. The dynamic programming principle and the connection between the value function and the viscosity solution of the associated Hamilton-Jacobi-Bellman equation are established in this setting by the generalized comparison theorem of backward stochastic …
In this paper, which is a continuation of the previously published discrete time paper we develop a theory for continuous time stochastic control problems which, in various ways, are time inconsistent in the sense that they do not admit a Bellman optimality principle. We study these problems within a game theoretic framework, and we look for Nash subgame perfect equilibrium points. For a general controlled continuous time Markov process and a fairly general objective functional we derive an extension of the standard Hamilton-Jacobi-Bellman equation, in the form of a system of non-linear equati…
This paper deals with a stochastic recursive optimal control problem, where the diffusion coefficient depends on the control variable and the control domain is not necessarily convex. We focus on the connection between the general maximum principle and the dynamic programming principle for such control problem without the assumption that the value is smooth enough, the set inclusions among the sub- and super-jets of the value function and the first-order and second-order adjoint processes as well as the generalized Hamiltonian function are established. Moreover, by comparing these results with…
This paper is concerned with solving chance-constrained finite-horizon optimal control problems, with a particular focus on the recursive feasibility issue of stochastic model predictive control (SMPC) in terms of mission-wide probability of safety (MWPS). MWPS assesses the probability that the entire state trajectory lies within the constraint set, and the objective of the SMPC controller is to ensure that it is no less than a threshold value. This differs from classic SMPC where the probability that the state lies in the constraint set is enforced independently at each time instant. Unlike r…
In this paper, the open-loop and closed-loop local and remote stochastic nonzero-sum game (LRSNG) problem is investigated. Different from previous works, the stochastic nonzero-sum game problem under consideration is essentially a special class of two-person nonzero-sum game problem, in which the information sets accessed by the two players are inconsistent. More specifically, both the local player and the remote player are involved in the system dynamics, and the information sets obtained by the two players are different, and each player is designed to minimize its own cost function. For the …
In this paper, we investigate a class of time-inconsistent discrete-time stochastic linear-quadratic optimal control problems, whose time-consistent solutions consist of an open-loop equilibrium control and a linear feedback equilibrium strategy. The open-loop equilibrium control is defined for a given initial pair, while the linear feedback equilibrium strategy is defined for all the initial pairs. Maximum-principle-type necessary and sufficient conditions containing stationary and convexity are derived for the existence of these two time-consistent solutions, respectively. Furthermore, for t…
In this paper, we consider a general time-inconsistent optimal control problem for a non homogeneous linear system, in which its state evolves according to a stochastic differential equation with deterministic coefficients, when the noise is driven by a Brownian motion and an independent Poisson point process. The running and the terminal costs in the objective functional, are explicitly dependent on some general discounting coefficients which cover the non-exponential and the hyperbolic discounting situations. Furthermore, the presence of some quadratic terms of the conditional expectation of…
We obtain the variational equations for backward stochastic differential equations in recursive stochastic optimal control problems, and then get the maximum principle which is novel. The control domain need not be convex, and the generator of the backward stochastic differential equation can contain z.
In this paper, we propose a novel equilibrium solution notion for the time-inconsistent stochastic linear-quadratic optimal control problem. This notion is called the mixed equilibrium solution, which consists of two parts: a pure-feedback-strategy part and an open-loop-control part. When the pure-feedback-strategy part is zero or the open-loop-control part does not depend on the initial state, the mixed equilibrium solution reduces to the open-loop equilibrium control and the feedback equilibrium strategy, respectively. Using a maximum-principle-like methodology with forward-backward stochast…
This paper is concerned with the open-loop time-consistent solution of time-inconsistent mean-field stochastic linear-quadratic optimal control. Different from standard stochastic linear-quadratic problems, both the system matrices and the weighting matrices are dependent on the initial times, and the conditional expectations of the control and state enter quadratically into the cost functional. Such features will ruin Bellman's principle of optimality and result in the time-inconsistency of the optimal control. Based on the dynamical nature of the systems involved, a kind of open-loop time-co…
In this paper, we study backward doubly stochastic recursive optimal control problem where the cost function is described by the solution of a backward doubly stochastic differential equation. We give the dynamical programming principle for this kind of optimal control problem and show that the value function is the unique Sobolev weak solution for the corresponding stochastic Hamilton-Jacobi-Bellman equation.
In this paper, we consider a stochastic recursive optimal control problem under model uncertainty. In this framework, the cost function is described by solutions of a family of backward stochastic differential equations. With the help of the linearization techniques and weak convergence methods, we derive the corresponding stochastic maximum principle. Moreover, a linear quadratic robust control problem is also studied.
In this paper we study the optimal stochastic control problem for stochastic differential systems reflected in a domain. The cost functional is a recursive one, which is defined via generalized backward stochastic differential equations developed by Pardoux and Zhang [20]. The value function is shown to be the unique viscosity solution to the associated Hamilton-Jacobi-Bellman equation, which is a fully nonlinear parabolic partial differential equation with a nonlinear Neumann boundary condition. For this, we also prove some new estimates for stochastic differential systems reflected in a doma…
We extend the construction of equilibria for linear-quadratic and mean-variance portfolio problems available in the literature to a large class of mean-field time-inconsistent stochastic control problems in continuous time. Our approach relies on a time discretization of the control problem via n-person games, which are characterized via the maximum principle using Backward Stochastic Differential Equations (BSDEs). The existence of equilibria is proved by applying weak convergence arguments to the solutions of n-person games. A numerical implementation is provided by approximating n-person ga…
In this paper, a Nash-type fictitious game framework is introduced to handle a time-inconsistent linear-quadratic optimal control. The Nash-type game in this framework is called fictitious as it is between the decision maker (called real player) and an auxiliary control variable (called fictitious player) with the real player and fictitious player looking for time-consistent policy and precomitted optimal policy, respectively. Namely, the fictitious-game framework is actually an auxiliary-variable-based mechanism where the fictitious player is our particular design. Noting that the real player…
On the Bellman equation in recursive stochastic dynamic programming with the CES aggregator
Faculty of Pure and Applied Mathematics, Wrocław University of Science and Technology（弗罗茨瓦夫科技大学纯应用数学学院）；Institute of Mathematics, University of Zielona Góra（热舒夫大学数学研究所）
In this paper, we study one kind of stochastic recursive optimal control problem with the obstacle constraints for the cost function where the cost function is described by the solution of one reflected backward stochastic differential equations. We will give the dynamic programming principle for this kind of optimal control problem and show that the value function is the unique viscosity solution of the obstacle problem for the corresponding Hamilton-Jacobi-Bellman equations.
The paper deals with a class of time-inconsistent control problems for McKean-Vlasov dynamics. By solving a backward time-inconsistent Hamilton-Jacobi-Bellman (HJB for short) equation coupled with a forward distribution-dependent stochastic differential equation, we investigate the existence and uniqueness of a closed-loop equilibrium for such time-inconsistent distribution-dependent control problem. Moreover, a special case of semi-linear McKean-Vlasov dynamics with a quadratic-type cost functional is considered due to its special structure.
Lehigh University（理海大学）；University of Oxford（牛津大学）
In this paper, we study and analyze the mini-batch version of StochAstic Recursive grAdient algoritHm (SARAH), a method employing the stochastic recursive gradient, for solving empirical loss minimization for the case of nonconvex losses. We provide a sublinear convergence rate (to stationary points) for general nonconvex functions and a linear convergence rate for gradient dominated functions, both of which have some advantages compared to other modern stochastic gradient algorithms for nonconvex losses.
In this paper, we study two kinds of singular optimal controls (SOCs for short) problems where the systems governed by forward-backward stochastic differential equations (FBSDEs for short), in which the control has two components: the regular control, and the singular one. Both drift and diffusion terms may involve the regular control variable. The regular control domain is postulated to be convex. Under certain assumptions, in the framework of the Malliavin calculus, we derive the pointwise second-order necessary conditions for stochastic SOC in the classical sense. This condition is describe…
We present AI-SARAH, a practical variant of SARAH. As a variant of SARAH, this algorithm employs the stochastic recursive gradient yet adjusts step-size based on local geometry. AI-SARAH implicitly computes step-size and efficiently estimates local Lipschitz smoothness of stochastic functions. It is fully adaptive, tune-free, straightforward to implement, and computationally efficient. We provide technical insight and intuitive illustrations on its design and convergence. We conduct extensive empirical analysis and demonstrate its strong performance compared with its classical counterparts and…
