In this paper, we study a stochastic linear-quadratic control problem with random coefficients and regime switching on a horizon $[0,T\wedgeτ]$, where $τ$ is a given random jump time for the underlying state process and $T$ is a constant. We obtain an explicit optimal state feedback control and explicit optimal cost value by solving a system of stochastic Riccati equations (SREs) with jumps on $[0,T\wedgeτ]$. By the decomposition approach stemming from filtration enlargement theory, we express the solution of the system of SREs with jumps in terms of another system of SREs involving only Brownian filtration on the deterministic horizon $[0,T]$. Solving the latter system is the key theoretical contribution of this paper and we establish this for three different cases, one of which seems to be new in the literature. These results are then applied to study a mean-variance hedging problem with random parameters that depend on both Brownian motion and Markov chain. The optimal portfolio and optimal value are presented in closed forms with the aid of a system of linear backward stochastic differential equations with jumps and unbounded coefficients in addition to the SREs with jumps.
This work focuses on stability analysis of numerical solutions to jump diffusions and jump diffusions with Markovian switching. Due to the use of Poisson processes, using asymptotic expansions as in the usual approach of treating diffusion processes does not work. Different from the existing treatments of Euler-Maurayama methods for solutions of stochastic differential equations, we use techniques from stochastic approximation. We analyze the almost sure exponential stability and exponential $p$-stability. The benchmark test model in numerical solutions, namely, one-dimensional linear scalar j…
This paper presents three versions of maximum principle for a stochastic optimal control problem of Markov regime-switching forward-backward stochastic differential equations with jumps (FBSDEJs). A general sufficient maximum principle for optimal control for a system driven by a Markov regime-switching forward and backward jump-diffusion model is developed. After, an equivalent maximum principle is proved. Malliavin calculus is also employed to derive a general stochastic maximum principle. The latter does not require concavity of Hamiltonian. Applications of the stochastic maximum principle …
Motivated by networked systems, stochastic control, optimization, and a wide variety of applications, this work is devoted to systems of switching jump diffusions. Treating such nonlinear systems, we focus on stability issues. First asymptotic stability in the large is obtained. Then the study on exponential p-stability is carried out. Connection between almost surely exponential stability and exponential p-stability is exploited. Also presented are smooth-dependence on the initial data. Using the smooth-dependence, necessary conditions for exponential p-stability are derived. Then criteria fo…
This work develops an approximation procedure for a class of non-zero-sum stochastic differential investment and reinsurance games between two insurance companies. Both proportional reinsurance and excess-of loss reinsurance policies are considered. We develop numerical algorithms to obtain the Nash equilibrium by adopting the Markov chain approximation methodology and applying the dynamical programming principle for the nonlinear integro-differential Hamilton-Jacobi-Isaacs (HJI) equations. Furthermore, we establish the convergence of the approximation sequences and the approximation to the va…
We prove maximum principles for the problem of optimal control for a jump diffusion with infinite horizon and partial information. The results are applied to partial information optimal consumption and portfolio problems in infinite horizon.
This paper is concerned with a discounted optimal control problem of partially observed forward-backward stochastic systems with jumps on infinite horizon. The control domain is convex and a kind of infinite horizon observation equation is introduced. The uniquely solvability of infinite horizon forward (backward) stochastic differential equation with jumps is obtained and more extended analysis, especially for the backward case, is made. Some new estimates are first given and proved for the critical variational inequality. Then an ergodic maximum principle is obtained by introducing some infi…
We consider an optimal switching problem with random lag and possibility of component failure. The random lag is modeled by letting the operation mode follow a regime switching Markov-model with transition intensities that depend on the switching mode. The possibility of failures is modeled by having absorbing components. We show existence of an optimal control for the problem by applying a probabilistic technique based on the concept of Snell envelopes.
We study a zero-sum stochastic differential switching game in infinite horizon. We prove the existence of the value of the game and characterize it as the unique viscosity solution of the associated system of quasi-variational inequalities with bilateral obstacles. We also obtain a verification theorem which provides an optimal strategy of the game. Finally, some numerical examples with two regimes are given.
In this work, we study the problem of mean-variance hedging with a random horizon T ^ tau, where T is a deterministic constant and is a jump time of the underlying asset price process. We rst formulate this problem as a stochastic control problem and relate it to a system of BSDEs with jumps. We then provide a veri cation theorem which gives the optimal strategy for the mean-variance hedging using the solution of the previous system of BSDEs. Finally, we prove that this system of BSDEs admits a solution via a decomposition approach coming from ltration enlargement theory.
We consider a jump-diffusion mean field control problem with regime switching in the state dynamics. The corresponding value function is characterized as the unique viscosity solution of a HJB master equation on the space of probability measures. Using this characterization, we prove that the value function, which is not regular, is the limit of a finite agent centralized optimal control problem as the number of agents go to infinity, with an explicit convergence rate. Assuming in addition that the value function is smooth, we establish a quantitative propagation of chaos result for the optima…
The Hybrid Minimum Principle (HMP) is established for the optimal control of deterministic hybrid systems with both autonomous and controlled switchings and jumps where state jumps at the switching instants are permitted to be accompanied by changes in the dimension of the state space. First order variational analysis is performed via the needle variation methodology and the necessary optimality conditions are established in the form of the HMP. A feature of special interest in this work is the explicit presentations of boundary conditions on the Hamiltonians and the adjoint processes before a…
This paper analyzes a class of impulse control problems for multi-dimensional jump diffusions in the finite time horizon. Following the basic mathematical setup from Stroock and Varadhan \cite{StroockVaradhan06}, this paper first establishes rigorously an appropriate form of Dynamic Programming Principle (DPP). It then shows that the value function is a viscosity solution for the associated Hamilton-Jacobi-Belleman (HJB) equation involving integro-differential operators. Finally, under additional assumptions that the jumps are of infinite activity but are of finite variation and that the diffu…
This paper is concerned with a stochastic linear-quadratic (LQ) optimal control problem on infinite time horizon, with regime switching, random coefficients, and cone control constraint. To tackle the problem, two new extended stochastic Riccati equations (ESREs) on infinite time horizon are introduced. The existence of the nonnegative solutions, in both standard and singular cases, is proved through a sequence of ESREs on finite time horizon. Based on this result and some approximation techniques, we obtain the optimal state feedback control and optimal value for the stochastic LQ problem exp…
This paper provides a geometrical derivation of the Hybrid Minimum Principle (HMP) for autonomous impulsive hybrid systems on Riemannian manifolds, i.e. systems where the manifold valued component of the hybrid state trajectory may have a jump discontinuity when the discrete component changes value. The analysis is expressed in terms of extremal trajectories on the cotangent bundle of the manifold state space. In the case of autonomous hybrid systems, switching manifolds are defined as smooth embedded submanifolds of the state manifold and the jump function is defined as a smooth map on the sw…
In this paper, we present some properties of time optimal controls for linear ODEs with the ball-type control constraint. More precisely, for an optimal control, we build up an upper bound for the number of its switching points; show that it jumps from one direction to the reverse direction at each switching point; give its dynamic behaviour between two consecutive switching points; and study its switching directions.
We consider an optimal switching problem where the terminal reward depends on the entire control trajectory. We show existence of an optimal control by applying a probabilistic technique based on the concept of Snell envelopes. We then apply this result to solve an impulse control problem for stochastic delay differential equations driven by a Brownian motion and an independent compound Poisson process. Furthermore, we show that the studied problem arises naturally when maximizing the revenue from operation of a group of hydro-power plants with hydrological coupling.
This paper studies a discrete-time optimal switching problem on a finite horizon. The underlying model has a running reward, terminal reward and signed (positive and negative) switching costs. Using the martingale approach to optimal stopping problems, we extend a well known explicit dynamic programming method for computing the value function and the optimal strategy to the case of signed switching costs.
This paper presents a novel framework which combines a non-iterative solution of Real-Time Nonlinear Receding Horizon Control (NRHC) methodology to achieve consensus within complex network topologies with existing time-delays and in presence of switching topologies. In this formulation, we solve the distributed nonlinear optimization problem for multi-agent network systems directly, \emph{in real-time}, without any dependency on iterative processes, where the stability and convergence guarantees are provided for the solution. Three benchmark examples on non-linear chaotic systems provide valid…
This paper mainly investigates the optimal control and stabilization problems for linear discrete-time Markov jump systems. The general case for the finite-horizon optimal controller is considered, where the input weighting matrix in the performance index is just required to be positive semi-definite. The necessary and sufficient condition for the existence of the optimal controller in finite-horizon is given explicitly from a set of coupled difference Riccati equations (CDRE). One of the key techniques is to solve the forward and backward stochastic difference equation (FDSDE) which is obtain…
We study local structure of time-optimal controls and trajectories for a 3-dimensional control-affine system with a 2-dimensional control parameter with values in the disk. In particular, we give sufficient conditions, in terms of Lie bracket relations, for optimal controls to be smooth or to have only isolated jump discontinuities.
In this paper we study continuous-time two-player zero-sum optimal switching games on a finite horizon. Using the theory of doubly reflected BSDEs with interconnected barriers, we show that this game has a value and an equilibrium in the players' switching controls.
This paper is concerned with optimal switching over multiple modes in continuous time and on a finite horizon. The performance index includes a running reward, terminal reward and switching costs that can belong to a large class of stochastic processes. Particularly, the switching costs are modelled by right-continuous with left-limits processes that are quasi-left-continuous and can take both positive and negative values. We provide sufficient conditions leading to a well known probabilistic representation of the value function for the switching problem in terms of interconnected Snell envelo…
This paper studies the time optimal control problem for systems of heat equations coupled by a pair of constant matrices. The control constraint is of the ball-type, while the target is the origin of the state space. We obtain an upper bound for the number of switching points of the optimal control over each interval with a fixed length. Also, we prove that at each switching point, the optimal control jump from one direction to the reverse direction.
Finite horizon stochastic $H_2/H_\infty$ control for continuous-time mean-field systems with Poisson jumps
Zhongtai Securities Institute for Financial Studies, Shandong University（山东大学中泰证券金融研究院）；College of Electrical Engineering and Automation, Shandong University of Science and Technology（山东科技大学电气工程学院）
In this paper we show existence and uniqueness of a solution for a system of m variational partial differential inequalities with inter-connected obstacles. This system is the deterministic version of the Verification Theorem of the Markovian optimal m-states switching problem. The switching cost functions are arbitrary. This problem is in relation with the valuation of firms in a financial market.
In this paper we analyse local regularity of time-optimal controls and trajectories for an n-dimensional affine control system with a control parameter, taking values in a k-dimensional closed ball. In the case of k equal to n-1, we give sufficient conditions in terms of Lie bracket relations for all optimal controls to be smooth or to have only isolated jump discontinuities.
We consider the problem of optimal multiple switching in finite horizon, when the state of the system, including the switching costs, is a general adapted stochastic process. The problem is formulated as an extended impulse control problem and completely solved using probabilistic tools such as the Snell envelop of processes and reflected backward stochastic differential equations. Finally, when the state of the system is a Markov diffusion process, we show that the vector of value functions of the optimal problem is a viscosity solution to a system of variational inequalities with inter-conne…
In this paper, we consider a continuous-time mean-variance portfolio selection with regime-switching and random horizon. Unlike previous works, the dynamic of assets are described by non-Markovian regime-switching models in the sense that all the market parameters are predictable with respect to the filtration generated jointly by Markov chain and Brownian motion. We formulate this problem as a constrained stochastic linear-quadratic optimal control problem. The Markov chain is assumed to be independent of the Brownian motion. So the market is incomplete. We derive closed-form expressions for …
In this paper, we investigate infinite horizon jump-diffusion forward-backward stochastic differential equations under some monotonicity conditions. We establish an existence and uniqueness theorem, two stability results and a comparison theorem for solutions to such kind of equations. Then the theoretical results are applied to study a kind of infinite horizon backward stochastic linear-quadratic optimal control problems, and then differential game problems. The unique optimal controls for the control problems and the unique Nash equilibrium points for the game problems are obtained in closed…
This paper uses recent results on continuous-time finite-horizon optimal switching problems with negative switching costs to prove the existence of a saddle point in an optimal stopping (Dynkin) game. Sufficient conditions for the game's value to be continuous with respect to the time horizon are obtained using recent results on norm estimates for doubly reflected backward stochastic differential equations. This theory is then demonstrated numerically for the special cases of cancellable call and put options in a Black-Scholes market.
We develop an approach for two player constraint zero-sum and nonzero-sum stochastic differential games, which are modeled by Markov regime-switching jump-diffusion processes. We provide the relations between a usual stochastic optimal control setting and a Lagrangian method. In this context, we prove corresponding theorems for two different type of constraints, which lead us to find real valued and stochastic Lagrange multipliers, respectively. Then, we illustrate our results for a nonzero-sum game problem with stochastic maximum principle technique. Our application is an example of cooperati…
This paper studies the problem of the deterministic version of the Verification Theorem for the optimal m-states switching in infinite horizon under Markovian framework with arbitrary switching cost functions. The problem is formulated as an extended impulse control problem and solved by means of probabilistic tools such as the Snell envelop of processes and reflected backward stochastic differential equations. A viscosity solutions approach is employed to carry out a finne analysis on the associated system of m variational inequalities with inter-connected obstacles. We show that the vector o…
This paper will investigate the infinite horizon optimal control and stabilization problems for the Markov jump linear system (MJLS) subject to control input delay. Different from previous works, for the first time, the necessary and sufficient stabilization conditions are explored under explicit expressions, and the optimal controller for infinite horizon is designed with a coupled algebraic Riccati equation. By introducing a new type of Lyapunov equation, we show that under the exact observability assumption, the MJLS with control input delay is stabilizable in the mean square sense with the…
