This paper is concerned with a general non-homogeneous stochastic linear quadratic (LQ) control problem with regime switching and random coefficients. We obtain the explicit optimal state feedback control and optimal value for this problem in terms of two systems of backward stochastic differential equations (BSDEs): one is the famous stochastic Riccati equation and the other one is a new linear multi-dimensional BSDE with all coefficients being unbounded. The existence and uniqueness of the solutions to these two systems of BSDEs are proved by means of BMO martingales and contraction mapping method. At last, the theory is applied to study an asset-liability management problem under the mean-variance criteria.
In this technical note we address the problem of achieving consensus in a network of homogeneous nonlinear systems. The communication network is supposed to be switching within a finite set of topologies which may be disconnected for finite time intervals. We prove that if the length of the time intervals in which connected topologies are active satisfy an average dwell-time condition consensus is achieved. Lyapunov arguments proposed in the field of hybrid nonlinear systems are adopted to prove the result.
This paper presents three versions of maximum principle for a stochastic optimal control problem of Markov regime-switching forward-backward stochastic differential equations with jumps (FBSDEJs). A general sufficient maximum principle for optimal control for a system driven by a Markov regime-switching forward and backward jump-diffusion model is developed. After, an equivalent maximum principle is proved. Malliavin calculus is also employed to derive a general stochastic maximum principle. The latter does not require concavity of Hamiltonian. Applications of the stochastic maximum principle …
We prove a sufficient stochastic maximum principle for the optimal control of a regime-switching diffusion model. We show the connection to dynamic programming and we apply the result to a quadratic loss minimization problem, which can be used to solve a mean-variance portfolio selection problem.
We study a zero-sum stochastic differential switching game in infinite horizon. We prove the existence of the value of the game and characterize it as the unique viscosity solution of the associated system of quasi-variational inequalities with bilateral obstacles. We also obtain a verification theorem which provides an optimal strategy of the game. Finally, some numerical examples with two regimes are given.
This paper considers a stochastic production planning problem with regime switching. There are two regimes corresponding to different economic cycles. A factory is planning its production so as to minimize production costs. We analyze this problem through the value function approach. The optimal production is characterized through the solution of an elliptic system of partial differential equations which is shown to have a solution.
In this paper, we present an optimal control problem for stochastic differential games under Markov regime-switching forward-backward stochastic differential equations with jumps and partial information. First, we prove a sufficient maximum principle for non zero-sum stochastic differential game problems and obtain equilibrium point for such games. Second, we prove an equivalent maximum principle for non zero-sum stochastic differential games. The zero-sum stochastic differential games equivalent maximum principle is then obtained as a corollary. We apply the obtained results to study a proble…
In this paper, we establish a general stochastic maximum principle for optimal control for systems described by a continuous-time Markov regime-switching stochastic recursive utilities model. The control domain is postulated not to be convex, and the diffusion terms depend on control variables. To this end, we first study a kind of classical forward stochastic optimal control problems. Afterwards, based on previous results, we introduce two groups of new first and second-order adjoint equations. The corresponding variational equations for forward-backward stochastic differential equations are …
This paper is concerned with a stochastic linear-quadratic (LQ) optimal control problem on infinite time horizon, with regime switching, random coefficients, and cone control constraint. To tackle the problem, two new extended stochastic Riccati equations (ESREs) on infinite time horizon are introduced. The existence of the nonnegative solutions, in both standard and singular cases, is proved through a sequence of ESREs on finite time horizon. Based on this result and some approximation techniques, we obtain the optimal state feedback control and optimal value for the stochastic LQ problem exp…
In this paper we prove a weak necessary and sufficient maximum principle for Markovian regime switching stochastic optimal control problems. Instead of insisting on the maximum condition of the Hamiltonian, we show that 0 belongs to the sum of Clarke's generalized gradient of the Hamiltonian and Clarke's normal cone of the control constraint set at the optimal control. Under a joint concavity condition on the Hamiltonian and a convexity condition on the terminal objective function, the necessary condition becomes sufficient. We give four examples to demonstrate the weak stochastic maximum prin…
This paper investigates the distributed event-based consensus problem of switching networks satisfying the jointly connected condition. Both the state consensus of homogeneous linear networks and output consensus of heterogeneous networks are studied. Two kinds of event-based protocols based on local sampled information are designed, without the need to solve any matrix equation or inequality. Theoretical analysis indicates that the proposed event-based protocols guarantee the achievement of consensus and the exclusion of Zeno behaviors for jointly connected undirected switching graphs. These …
This paper investigates the stochastic linear quadratic (LQ, for short) optimal control problem of Markov regime switching system. The representation of the cost functional for the stochastic LQ optimal control problem of Markov regime switching system is derived using the technique of It{ô}'s formula. For the stochastic LQ optimal control problem of Markov regime switching system, we establish the equivalence between the open-loop (closed-loop) solvability and the existence of an adapted solution to the corresponding forward-backward stochastic differential equation with constraint (the exist…
The optimization of process economics within the model predictive control (MPC) formulation has given rise to a new control paradigm known as economic MPC (EMPC). Several authors have discussed the closed-loop properties of EMPC-controlled deterministic systems, however, little have uncertain systems been studied. In this paper we propose EMPC formulations for nonlinear Markovian switching systems which guarantee recursive feasibility, asymptotic performance bounds and constrained mean square (MS) stability.
This paper first describes a class of uncertain stochastic control systems with Markovian switching, and derives an Itô-Liu formula for Markov-modulated processes. And we characterize an optimal control law, which satisfies the generalized Hamilton-Jacobi-Bellman (HJB) equation with Markovian switching. Then, by using the generalized HJB equation, we deduce the optimal consumption and portfolio policies under uncertain stochastic financial markets with Markovian switching. Finally, for constant relative risk-aversion (CRRA) felicity functions, we explicitly obtain the optimal consumption and p…
In this paper, we consider a linear quadratic (LQ) leader-follower stochastic differential game for regime switching diffusions with mean-field interactions. One of the salient features of this paper is that conditional mean-field terms are included in the state equation and cost functionals. Based on stochastic maximum principles (SMPs), the follower's problem and the leader's problem are solved sequentially and an open-loop Stackelberg equilibrium is obtained. Further, with the help of the so-called four-step scheme, the corresponding Hamiltonian systems for the two players are decoupled and…
In this paper we set up an optimal control framework for a hybrid stochastic system with dual or multiple Markov switching diffusion processes, while Markov chains governing these switching diffusions are not identical as assumed by the existing literature. As an application and illustration of this model, we solve a portfolio choice problem for an investor facing financial and labor markets that are both regime switching. In continuous time context we combine two separate Markov chains into one synthetic Markov chain and derive its corresponding generator matrix, then state the HJB equations …
In this work we provide explicit conditions on the existence of optimal feedback controls for stochastic processes with regime-switching. We use the compactification method which needs less regularity conditions on the coefficients of the studied stochastic systems. Two different kinds of controls have been considered: one is the control on the coefficients of the diffusion processes, another is the control on the transition rate matrices of the continuous-time Markov chains. Moreover, the dynamic programming principle is established after showing the continuity of the value function.
An optimal control problem is considered for a stochastic differential equation containing a state-dependent regime switching, with a recursive cost functional. Due to the non-exponential discounting in the cost functional, the problem is time-inconsistent in general. Therefore, instead of finding a global optimal control (which is not possible), we look for a time-consistent (approximately) locally optimal equilibrium strategy. Such a strategy can be represented through the solution to a system of partial differential equations, called an equilibrium Hamilton-Jacob-Bellman (HJB, for short) eq…
In this paper, we investigate the open-loop and weak closed-loop solvabilities of stochastic linear quadratic (LQ, for short) optimal control problem of Markovian regime switching system. Interestingly, these two solvabilities are equivalent. We first provide an alternative characterization of the open-loop solvability of the LQ problem using the perturbation approach. Then, we study the weak closed-loop solvability of the LQ problem of Markovian regime switching system, and establish the equivalent relationship between open-loop and weak closed-loop solvabilities. Finally, we present an examp…
We present an emergent stochastic flocking dynamics of the Cucker-Smale (CS) ensemble under randomly switching topologies. The evolution of the CS ensemble with randomly switching topologies involves two random components (switching times and choices of network topologies at switching instant). First, we allow switching times for the network topology to be random so that the successive increments are i.i.d. processes following the common probability distribution. Second, at each switching instant, we choose a network topology randomly from a finite set of admissible network topologies whose un…
Multidimensional indefinite stochastic Riccati equations and zero-sum stochastic linear-quadratic differential games with non-Markovian regime switching
This paper thoroughly investigates stochastic linear-quadratic optimal control problems with the Markovian regime switching system, where the coefficients of the state equation and the weighting matrices of the cost functional are random. We prove the solvability of the stochastic Riccati equation under the uniform convexity condition and obtain the closed-loop representation of the open-loop optimal control using the unique solvability of the corresponding stochastic Riccati equation. Moreover, by applying Itô's formula with jumps, we get a representation of the cost functional on a Hilbert s…
This paper studies a system of $m$ variational inequalities with interconnected obstacles in infinite horizon associated to optimal multi-modes switching problems. Our main result is the existence and uniqueness of a continuous solution in viscosity sense, for that system. The proof of the main result strongly relies on the connection between the systems of variational inequalities and reflected backward stochastic differential equations (RBSDEs) with oblique reflection, which will be characterized through a Feynman-Kac's formula. The main feature of our system of infinite horizon RBSDEs is th…
We address the problem of making a managerial decision when the investment project is subsidized, which results in the resolution of an infinite-horizon optimal stopping problem of a switching diffusion driven by either an homogeneous or an inhomogeneous continuous-time Markov chain. We provide a characterization of the value function (and optimal strategy) of the optimal stopping problem. On the one hand, broadly, we can prove that the value function is the unique viscosity solution to a system of HJB equations. On the other hand, when the Markov chain is homogeneous and the switching diffusi…
Stabilization of highly nonlinear hybrid stochastic differential delay equations by periodically intermittent feedback controls based on discrete-time observations with asynchronous switching
School of Science, Beijing University of Chemical Technology（北京化工大学理学院）
We study optimal behavior of energy producers under a CO_2 emission abatement program. We focus on a two-player discrete-time model where each producer is sequentially optimizing her emission and production schedules. The game-theoretic aspect is captured through a reduced-form price-impact model for the CO_2 allowance price. Such duopolistic competition results in a new type of a non-zero-sum stochastic switching game on finite horizon. Existence of game Nash equilibria is established through generalization to randomized switching strategies. No uniqueness is possible and we therefore conside…
This paper is concerned with the controller-and-stopper stochastic differential game under a regime switching model in an infinite horizon. The state of the system consists of a number of diffusions \emph{coupled} by a continuous-time finite-state Markov chain. There are two players, one called the controller and the other called the stopper, involved in the game. The goal is to find a saddle point for the two players up to the time that the stopper \emph{terminates} the game. Based on the dynamic programming principle (DPP, for short), the lower and upper value functions are shown to be the v…
This paper is concerned with a stochastic linear-quadratic optimal control problem with regime switching, random coefficients, and cone control constraint. The randomness of the coefficients comes from two aspects: the Brownian motion and the Markov chain. Using Itô's lemma for Markov chain, we obtain the optimal state feedback control and optimal cost value explicitly via two new systems of extended stochastic Riccati equations (ESREs). We prove the existence and uniqueness of the two ESREs using tools including multidimensional comparison theorem, truncation function technique, log transform…
A Convergent Algorithm Based on Deterministic Approximation for a Large Class of Regime-Switching Generalized Stochastic Game-Theoretic Riccati Differential Equations
In this paper, we study a stochastic linear-quadratic control problem with random coefficients and regime switching on a horizon $[0,T\wedgeτ]$, where $τ$ is a given random jump time for the underlying state process and $T$ is a constant. We obtain an explicit optimal state feedback control and explicit optimal cost value by solving a system of stochastic Riccati equations (SREs) with jumps on $[0,T\wedgeτ]$. By the decomposition approach stemming from filtration enlargement theory, we express the solution of the system of SREs with jumps in terms of another system of SREs involving only Brown…
We study a finite-dimensional continuous-time optimal control problem on finite horizon for a controlled diffusion driven by Brownian motion, in the linear-quadratic case. We admit stochastic coefficients, possibly depending on an underlying independent marked point process, so that our model is general enough to include controlled switching systems where the switching mechanism is not required to be Markovian. The problem is solved by means of a Riccati equation, which a backward stochastic differential equation driven by the Bronwian motion and by the random measure associated to the marked …
In this paper we provide a novel approach to the analysis of kinetic models for label switching, which are used for particle systems that can randomly switch between gradient flows in different energy landscapes. Besides problems in biology and physics, we also demonstrate that stochastic gradient descent, the most popular technique in machine learning, can be understood in this setting, when considering a time-continuous variant. Our analysis is focusing on the case of evolution in a collection of external potentials, for which we provide analytical and numerical results about the evolution a…
We consider the problem of optimal multiple switching in finite horizon, when the state of the system, including the switching costs, is a general adapted stochastic process. The problem is formulated as an extended impulse control problem and completely solved using probabilistic tools such as the Snell envelop of processes and reflected backward stochastic differential equations. Finally, when the state of the system is a Markov diffusion process, we show that the vector of value functions of the optimal problem is a viscosity solution to a system of variational inequalities with inter-conne…
This paper is devoted to a global stochastic maximum principle for conditional mean-field forward-backward stochastic differential equations (FBSDEs, for short) with regime switching. The control domain is unnecessarily convex and the driver of backward stochastic differential equations (BSDEs, for short) could depend on $Z$. Different from the case of non-recursive utility, the first-order and second-order adjoint equations are both high-dimensional linear BSDEs. Based on the adjoint equations, we reveal the relations among the terms of the first- and second-order Taylor's expansions. A gener…
A discrete-time stochastic optimal control problem was recently proposed to address the GLOSA (Green Light Optimal Speed Advisory) problem in cases where the next signal switching time is decided in real time and is therefore uncertain in advance. The corresponding numerical solution via SDP (Stochastic Dynamic Programming) calls for substantial computation time, which excludes problem solution in the vehicle's on-board computer in real time. To overcome the computation time bottleneck, as a first attempt, a modified version of Dynamic Programming, known as Discrete Differential Dynamic Progra…
