In this paper, we formulate a general time-inconsistent stochastic linear--quadratic (LQ) control problem. The time-inconsistency arises from the presence of a quadratic term of the expected state as well as a state-dependent term in the objective functional. We define an equilibrium, instead of optimal, solution within the class of open-loop controls, and derive a sufficient condition for equilibrium controls via a flow of forward--backward stochastic differential equations. When the state is one dimensional and the coefficients in the problem are all deterministic, we find an explicit equilibrium control. As an application, we then consider a mean-variance portfolio selection model in a complete financial market where the risk-free rate is a deterministic function of time but all the other market parameters are possibly stochastic processes. Applying the general sufficient condition, we obtain explicit equilibrium strategies when the risk premium is both deterministic and stochastic.
A time-inconsistent optimal control problem is formulated and studied for a controlled linear ordinary differential equation with quadratic cost functional. A notion of equilibrium control is introduced, which can be regarded as a time-consistent solution to the original time-inconsistent problem. Under certain conditions, we constructively prove the existence of such an equilibrium control which is represented via a forward ordinary differential equation coupled with a backward Riccati--Volterra integral equation. Our constructive approach is based on the introduction of a family of $N$-perso…
This paper deals with a class of time inconsistent stochastic linear quadratic (SLQ) optimal control problems in Markovian framework. Three notions, i.e., closed-loop equilibrium controls/strategies, open-loop equilibrium controls and their closed-loop representations, are characterized in unified manners. These results indicate clearer and deeper distinctions among these notions. For example, in particular time consistent setting, the open-loop equilibrium controls are fully characterized by first-order, second-order necessary optimality conditions, and become needlessly optimal, while the cl…
In this paper, we study extended backward stochastic Volterra integral equations (EBSVIEs, for short). We establish the well-posedness under weaker assumptions than the literature, and prove a new kind of regularity property for the solutions. As an application, we investigate, in the open-loop framework, a time-inconsistent stochastic recursive control problem where the cost functional is defined by the solution to a backward stochastic Volterra integral equation (BSVIE, for short). We show that the corresponding adjoint equations become EBSVIEs, and provide a necessary and sufficient conditi…
We extend the construction of equilibria for linear-quadratic and mean-variance portfolio problems available in the literature to a large class of mean-field time-inconsistent stochastic control problems in continuous time. Our approach relies on a time discretization of the control problem via n-person games, which are characterized via the maximum principle using Backward Stochastic Differential Equations (BSDEs). The existence of equilibria is proved by applying weak convergence arguments to the solutions of n-person games. A numerical implementation is provided by approximating n-person ga…
In this paper, a class of time inconsistent linear quadratic optimal control problems of mean-field stochastic differential equations (SDEs) is considered under Markovian framework. Open-loop equilibrium controls and their particular closed-loop representations are introduced and characterized via variational ideas. Several interesting features are revealed and a system of coupled Riccati equations is derived. In contrast with the analogue optimal control problems of SDEs, the mean-field terms in state equation, which is another reason of time inconsistency, prompts us to define above two noti…
In this paper, we study a class of stochastic time-inconsistent linear-quadratic (LQ) control problems with control input constraints. These problems are investigated within the more general framework associated with random coefficients. This paper aims to further develop a new methodology, which fundamentally differs from those in the standard control (without constraints) theory in the literature, to cope with the mathematical difficulties raised due to the presence of input constraints. We first prove that the existence of an equilibrium solution is equivalent to the existence of a solution…
We present a numerical method for the minimization of constrained optimization problems where the objective is augmented with large quadratic penalties of inconsistent equality constraints. Such objectives arise from quadratic integral penalty methods for the direct transcription of optimal control problems. The Augmented Lagrangian Method (ALM) has a number of advantages over the Quadratic Penalty Method (QPM). However, if the equality constraints are inconsistent, then ALM might not converge to a point that minimizes the bias of the objective and penalty term. Therefore, we present a modific…
In this paper, we study a time-inconsistent stochastic optimal control problem with a recursive cost functional by a multi-person hierarchical differential game approach. An equilibrium strategy of this problem is constructed and a corresponding equilibrium Hamilton-Jacobi-Bellman (HJB, for short) equation is established to characterize the associated equilibrium value function. Moreover, a well-posedness result of the equilibrium HJB equation is established under certain conditions.
In this paper, we investigate a class of time-inconsistent discrete-time stochastic linear-quadratic optimal control problems, whose time-consistent solutions consist of an open-loop equilibrium control and a linear feedback equilibrium strategy. The open-loop equilibrium control is defined for a given initial pair, while the linear feedback equilibrium strategy is defined for all the initial pairs. Maximum-principle-type necessary and sufficient conditions containing stationary and convexity are derived for the existence of these two time-consistent solutions, respectively. Furthermore, for t…
This paper studies open-loop equilibriums for a general class of time-inconsistent stochastic control problems under jump-diffusion SDEs with deterministic coefficients. Inspired by the idea of Four-Step-Scheme for forward-backward stochastic differential equations with jumps (FBSDEJs, for short), we derive two systems of integro-partial differential equations (IPDEs, for short). Then, we rigorously prove a verification theorem which provides a sufficient condition for open-loop equilibrium strategies. As an illustration of the general theory, we discuss a mean-variance portfolio selection pro…
An optimal control problem is considered for a stochastic differential equation with the cost functional determined by a backward stochastic Volterra integral equation (BSVIE, for short). This kind of cost functional can cover the general discounting (including exponential and non-exponential) situation with a recursive feature. It is known that such a problem is time-inconsistent in general. Therefore, instead of finding a global optimal control, we look for a time-consistent locally near optimal equilibrium strategy. With the idea of multi-person differential games, a family of approximate e…
An optimal control problem is considered for a stochastic differential equation containing a state-dependent regime switching, with a recursive cost functional. Due to the non-exponential discounting in the cost functional, the problem is time-inconsistent in general. Therefore, instead of finding a global optimal control (which is not possible), we look for a time-consistent (approximately) locally optimal equilibrium strategy. Such a strategy can be represented through the solution to a system of partial differential equations, called an equilibrium Hamilton-Jacob-Bellman (HJB, for short) eq…
In this technical note, we consider the linear-quadratic time-inconsistent mean-field type leader-follower Stackelberg differential game with an adapted open-loop information structure. The objective functionals of the leader and the follower include conditional expectations of state and control (mean field) variables, and the cost parameters could be general nonexponential discounting depending on the initial time. As stated in the existing literature, these two general settings of the objective functionals induce time inconsistency in the optimal solutions. Given an arbitrary control of the …
We develop a theory for continuous-time non-Markovian stochastic control problems which are inherently time-inconsistent. Their distinguishing feature is that the classical Bellman optimality principle no longer holds. Our formulation is cast within the framework of a controlled non-Markovian forward stochastic differential equation, and a general objective functional setting. We adopt a game-theoretic approach to study such problems, meaning that we seek for sub-game perfect Nash equilibrium points. As a first novelty of this work, we introduce and motivate a refinement of the definition of e…
In this paper, which is a continuation of the previously published discrete time paper we develop a theory for continuous time stochastic control problems which, in various ways, are time inconsistent in the sense that they do not admit a Bellman optimality principle. We study these problems within a game theoretic framework, and we look for Nash subgame perfect equilibrium points. For a general controlled continuous time Markov process and a fairly general objective functional we derive an extension of the standard Hamilton-Jacobi-Bellman equation, in the form of a system of non-linear equati…
A general time-inconsistent optimal control problem is considered for stochastic differential equations with deterministic coefficients. Under suitable conditions, a Hamilton-Jacobi-Bellman type equation is derived for the equilibrium value function of the problem. Well-posedness and some properties of such an equation is studied, and time-consistent equilibrium strategies are constructed. As special cases, the linear-quadratic problem and a generalized Merton's portfolio problem are investigated.
In this paper, a Nash-type fictitious game framework is introduced to handle a time-inconsistent linear-quadratic optimal control. The Nash-type game in this framework is called fictitious as it is between the decision maker (called real player) and an auxiliary control variable (called fictitious player) with the real player and fictitious player looking for time-consistent policy and precomitted optimal policy, respectively. Namely, the fictitious-game framework is actually an auxiliary-variable-based mechanism where the fictitious player is our particular design. Noting that the real player…
The paper deals with a class of time-inconsistent control problems for McKean-Vlasov dynamics. By solving a backward time-inconsistent Hamilton-Jacobi-Bellman (HJB for short) equation coupled with a forward distribution-dependent stochastic differential equation, we investigate the existence and uniqueness of a closed-loop equilibrium for such time-inconsistent distribution-dependent control problem. Moreover, a special case of semi-linear McKean-Vlasov dynamics with a quadratic-type cost functional is considered due to its special structure.
In this paper, the open-loop and closed-loop local and remote stochastic nonzero-sum game (LRSNG) problem is investigated. Different from previous works, the stochastic nonzero-sum game problem under consideration is essentially a special class of two-person nonzero-sum game problem, in which the information sets accessed by the two players are inconsistent. More specifically, both the local player and the remote player are involved in the system dynamics, and the information sets obtained by the two players are different, and each player is designed to minimize its own cost function. For the …
In this paper, we consider a general time-inconsistent optimal control problem for a non homogeneous linear system, in which its state evolves according to a stochastic differential equation with deterministic coefficients, when the noise is driven by a Brownian motion and an independent Poisson point process. The running and the terminal costs in the objective functional, are explicitly dependent on some general discounting coefficients which cover the non-exponential and the hyperbolic discounting situations. Furthermore, the presence of some quadratic terms of the conditional expectation of…
In this paper, we propose a novel equilibrium solution notion for the time-inconsistent stochastic linear-quadratic optimal control problem. This notion is called the mixed equilibrium solution, which consists of two parts: a pure-feedback-strategy part and an open-loop-control part. When the pure-feedback-strategy part is zero or the open-loop-control part does not depend on the initial state, the mixed equilibrium solution reduces to the open-loop equilibrium control and the feedback equilibrium strategy, respectively. Using a maximum-principle-like methodology with forward-backward stochast…
This paper is concerned with the open-loop time-consistent solution of time-inconsistent mean-field stochastic linear-quadratic optimal control. Different from standard stochastic linear-quadratic problems, both the system matrices and the weighting matrices are dependent on the initial times, and the conditional expectations of the control and state enter quadratically into the cost functional. Such features will ruin Bellman's principle of optimality and result in the time-inconsistency of the optimal control. Based on the dynamical nature of the systems involved, a kind of open-loop time-co…
In this paper we study a class of time-inconsistent terminal Markovian control problems in discrete time subject to model uncertainty. We combine the concept of the sub-game perfect strategies with the adaptive robust stochastic to tackle the theoretical aspects of the considered stochastic control problem. Consequently, as an important application of the theoretical results, by applying a machine learning algorithm we solve numerically the mean-variance portfolio selection problem under the model uncertainty.
This paper is concerned with the existence and uniqueness of weak solutions to the Cauchy-Dirichlet problem of backward stochastic partial differential equations (BSPDEs) with nonhomogeneous terms of quadratic growth in both the gradient of the first unknown and the second unknown. As an example, we consider a non-Markovian stochastic optimal control problem with cost functional formulated by a quadratic BSDE, where the corresponding value function satisfies the above quadratic BSPDE.
We study a stochastic optimal control problem for forward-backward control systems with quadratic generators. In order to establish the first and second-order variational and adjoint equations, we obtain a new estimate for one-dimensional linear BSDEs with unbounded stochastic Lipschitz coefficients involving bounded mean oscillation martingales (BMO-martingales for short) and prove the solvability for a class of multi-dimensional BSDEs with this type. Finally, a new global stochastic maximum principle is deduced.
A linear quadratic optimal stochastic control problem with random coefficients and indefinite state/control weight costs is usually linked to an indefinite stochastic Riccati equation (SRE) which is a matrix-valued quadratic backward stochastic differential equation along with an algebraic constraint involving the unknown. Either the optimal control problem or the SRE is solvable only if the given data satisfy a certain structure condition that has yet to be precisely defined. In this paper, by introducing a notion of subsolution for the SRE, we derive several novel sufficient conditions for t…
A Linear-quadratic optimal control problem is considered for mean-field stochastic differential equations with deterministic coefficients. By a variational method, the optimality system is derived, which turns out to be a linear mean-field forward-backward stochastic differential equation. Using a decoupling technique, two Riccati differential equations are obtained, which are uniquely solvable under certain conditions. Then a feedback representation is obtained for the optimal control.
We study the closed-loop solvability of a stochastic linear quadratic optimal control problem for systems governed by stochastic evolution equations. This solvability is established by means of solvability of the corresponding Riccati equation, which is implied by the uniform convexity of the quadratic cost functional. At last, conditions ensuring the uniform convexity of the cost functional are discussed.
We present a numerical method for the minimization of objectives that are augmented with large quadratic penalties of overdetermined inconsistent equality constraints. Such objectives arise from quadratic integral penalty methods for the direct transcription of equality constrained optimal control problems. The Augmented Lagrangian Method (ALM) has a number of advantages over the Quadratic Penalty Method (QPM) for solving this class of problems. However, if the equality constraints of the discretization are inconsistent, then ALM might not converge to a point that minimizes the unconstrained b…
We present a novel method for solving a class of time-inconsistent optimal stopping problems by reducing them to a family of standard stochastic optimal control problems. In particular, we convert an optimal stopping problem with a non-linear function of the expected stopping time in the objective into optimization over an auxiliary value function for a standard stochastic control problem with an additional state variable. This approach differs from the previous literature which primarily employs Lagrange multiplier methods or relies on exact solutions. In contrast, we characterize the auxilia…
Linear-quadratic optimal control problems are considered for mean-field stochastic differential equations with deterministic coefficients. Time-inconsistency feature of the problems is carefully investigated. Both open-loop and closed-loop equilibrium solutions are presented for such kind of problems. Open-loop solutions are presented by means of variational method with decoupling of forward-backward stochastic differential equations, which lead to a Riccati equation system lack of symmetry. Closed-loop solutions are presented by means of multi-person differential games, the limit of which lea…
It is a longstanding unsolved problem to characterize the optimal feedbacks for general SLQs (i.e., stochastic linear quadratic control problems) with random coefficients in infinite dimensions; while the same problem but in finite dimensions was just addressed in a recent work [36]. This paper is devoted to giving a solution to this problem under some assumptions which can be verified for several interesting concrete models. More precisely, under these assumptions, we establish the equivalence between the existence of optimal feedback operator for infinite dimensional SLQs and the solvability…
A new definition of continuous-time equilibrium controls is introduced. As opposed to the standard definition, which involves a derivative-type operation, the new definition parallels how a discrete-time equilibrium is defined, and allows for unambiguous economic interpretation. The terms "strong equilibria" and "weak equilibria" are coined for controls under the new and the standard definitions, respectively. When the state process is a time-homogeneous continuous-time Markov chain, a careful asymptotic analysis gives complete characterizations of weak and strong equilibria. Thanks to Kakutan…
This paper investigates the stochastic linear quadratic (LQ, for short) optimal control problem of Markov regime switching system. The representation of the cost functional for the stochastic LQ optimal control problem of Markov regime switching system is derived using the technique of It{ô}'s formula. For the stochastic LQ optimal control problem of Markov regime switching system, we establish the equivalence between the open-loop (closed-loop) solvability and the existence of an adapted solution to the corresponding forward-backward stochastic differential equation with constraint (the exist…
We propose a simple and original approach for solving linear-quadratic mean-field stochastic control problems. We study both finite-horizon and infinite-horizon problems, and allow notably some coefficients to be stochastic. Our method is based on a suitable extension of the martingale formulation for verification theorems in control theory. The optimal control involves the solution to a system of Riccati ordinary differential equations and to a linear mean-field backward stochastic differential equation, existence and uniqueness conditions are provided for such a system. Finally, we illustrat…
In this paper, we consider linear quadratic optimal control with mean-field type for discrete-time stochastic systems with state and control dependent noise. An optimal control problem is studied for a linear mean-field stochastic differential equation with a quadratic cost functional. The coefficients and the weighting matrices in the cost functional are all assumed to be deterministic.
We consider a multi-player stochastic differential game with linear McKean-Vlasov dynamics and quadratic cost functional depending on the variance and mean of the state and control actions of the players in open-loop form. Finite and infinite horizon problems with possibly some random coefficients as well as common noise are addressed. We propose a simple direct approach based on weak martingale optimality principle together with a fixed point argument in the space of controls for solving this game problem. The Nash equilibria are characterized in terms of systems of Riccati ordinary different…
This paper considers the stochastic linear quadratic optimal control problem in which the control domain is nonconvex. By the functional analysis and convex perturbation methods, we establish a novel maximum principle. The application of the proposed maximum principle is illustrated through a work-out example.
In this paper, we investigate infinite horizon jump-diffusion forward-backward stochastic differential equations under some monotonicity conditions. We establish an existence and uniqueness theorem, two stability results and a comparison theorem for solutions to such kind of equations. Then the theoretical results are applied to study a kind of infinite horizon backward stochastic linear-quadratic optimal control problems, and then differential game problems. The unique optimal controls for the control problems and the unique Nash equilibrium points for the game problems are obtained in closed…
One of the fundamental issues in Control Theory is to design feedback controls. It is well-known that, the purpose of introducing Riccati equations in the deterministic case is to provide the desired feedback controls for linear quadratic control problems. To date, the same problem in the stochastic setting is only partially well-understood. In this paper, we establish the equivalence between the existence of optimal feedback controls for the stochastic linear quadratic control problems with random coefficients and the solvability of the corresponding backward stochastic Riccati equations in a…
