We consider impulse control of stochastic functional differential equations (SFDEs) driven by Lévy processes under an additional $L^p$-Lipschitz condition on the coefficients. Our results, which are first derived for a general stochastic optimization problem over infinite horizon impulse controls and then applied to the case of a controlled SFDE, apply to the infinite horizon as well as the random horizon settings. The methodology employed to show existence of optimal controls is a probabilistic one based on the concept of Snell envelopes.
In this work we show that one can solve a finite horizon non-Markovian impulse control problem with control dependant dynamics. This dynamic satisfies certain functional Lipschitz conditions and is path dependent in such a way that the resulting trajectory becomes a flow.
We consider the problem of impulse control minimax in finite horizon, when cost functions $(C(t,x,ξ)>0)$. We show existence of value function of the problem. Moreover, the value function is characterized as the unique viscosity solution of an Isaacs quasi-variational inequality. This problem is in relation with an application in mathematical finance.
We consider a two-player zero-sum deterministic differential game where each player uses both continuous and impulse controls in infinite-time horizon. We assume that the impulses supposed to be of general term and the costs depend on the state of the system. We use the dynamic programming principle and viscosity solutions approach to show existence and uniqueness of a solution for the Hamilton-Jacobi-Bellman-Isaacs (HJBI) partial differential equations (PDEs) of the game. We prove under Isaacs condition that the upper and lower value functions coincide.
We consider a robust impulse control problem in finite horizon where the underlying uncertainty stems from an impulsively and continuously controlled functional stochastic differential equation (FSDE) driven by Brownian motion. We assume that the controller acts upon the system by impulses while the adversary player (nature) acts through continuous controls. We look for a weak solution which leads us to consider a system of sequentially interconnected, obliquely reflected backward stochastic differential equations (RBSDEs) with stochastic Lipschitz coefficients. We show existence of solutions …
We study here the impulse control minimax problem. We allow the cost functionals and dynamics to be unbounded and hence the value functions can possibly be unbounded. We prove that the value function of the problem is continuous. Moreover, the value function is characterized as the unique viscosity solution of an Isaacs quasi-variational inequality. This problem is in relation with an application in mathematical finance.
In the present paper, we study a two-player zero-sum deterministic differential game with both players adopting impulse controls, in infinite time horizon, under rather weak assumptions on the cost functions. We prove by means of the dynamic programming principle (DPP) that the lower and upper value functions are continuous and viscosity solutions to the corresponding Hamilton-Jacobi-Bellman-Isaacs (HJBI) quasi-variational inequality (QVI). We define a new HJBI QVI for which, under a proportional property assumption on the maximizer cost, the value functions are the unique viscosity solution. …
This paper analyzes a class of impulse control problems for multi-dimensional jump diffusions in the finite time horizon. Following the basic mathematical setup from Stroock and Varadhan \cite{StroockVaradhan06}, this paper first establishes rigorously an appropriate form of Dynamic Programming Principle (DPP). It then shows that the value function is a viscosity solution for the associated Hamilton-Jacobi-Belleman (HJB) equation involving integro-differential operators. Finally, under additional assumptions that the jumps are of infinite activity but are of finite variation and that the diffu…
We consider stochastic impulse control problems when the impulses cost functions are arbitrary. We use the dynamic programming principle and viscosity solutions approach to show that the value function is a unique viscosity solution for the associated Hamilton-Jacobi-Bellman equation (HJB) partial differential equation (PDE) of stochastic impulse control problems
This paper studies the exponential stabilization on infinite dimensional system with impulse controls, where impulse instants appear periodically. The first main result shows that exponential stabilizability of the control system with a periodic feedback law is equivalent to one kind of weak observability inequalities. The second main result presents that, in the setting of a discrete LQ problem, the exponential stabilizability of control system with a periodic feedback law is equivalent to the solvability of an algebraic Riccati-type equation which was built up in [Qin, Wang and Yu, SIAM J. C…
This paper studies an optimal stochastic impulse control problem in a finite horizon with a decision lag, by which we mean that after an impulse is made, a fixed number units of time has to be elapsed before the next impulse is allowed to be made. The continuity of the value function is proved. A suitable version of dynamic programming principle is established, which takes into account the dependence of state process on the elapsed time. The corresponding Hamilton-Jacobi-Bellman (HJB) equation is derived, which exhibit some special feature of the problem. The value function of this optimal imp…
This paper considers the problem of two-player zero-sum stochastic differential game with both players adopting impulse controls in finite horizon under rather weak assumptions on the cost functions ($c$ and $χ$ not decreasing in time). We use the dynamic programming principle and viscosity solutions approach to show existence and uniqueness of a solution for the Hamilton-Jacobi-Bellman-Isaacs (HJBI) partial differential equation (PDE) of the game. We prove that the upper and lower value functions coincide.
Regularity of the impulse control problem for a non-degenerate $n$-dimensional jump diffusion with infinite activity and finite variation jumps was recently examined by Davis, Guo, and Wu (SICON 2010). Here we extend the analysis to include infinite activity and infinite variation jumps. More specifically, we show that the value function $u$ of the impulse control problem satisfies $u\in W^{2,p}_{\mathrm{loc}}(\mathbb{R}^{n})$.
We consider an impulse control problem in infinite horizon. To solve this problem, we extend to the infinite horizon case results of double barrier reflected backward stochastic differential equations. The properties of the Snell envelope can reduce our problem to show the existence of a pair of measurable continuous processes.
This paper considers a new class of deterministic finite-time horizon, two-player, zero-sum differential games (DGs) in which the maximizing player is allowed to take continuous and impulse controls whereas the minimizing player is allowed to take impulse control only. We seek to approximate the value function, and to provide a verification theorem for this class of DGs. We first, by means of dynamic programming principle (DPP) in viscosity solution (VS) framework, characterize the value function as the unique VS to the related Hamilton-Jacobi-Bellman-Isaacs (HJBI) double-obstacle equation. Ne…
This paper studies regularity property of the value function for an infinite-horizon discounted cost impulse control problem, where the underlying controlled process is a multidimensional jump diffusion with possibly `infinite-activity' jumps. Surprisingly, despite these jumps, we obtain the same degree of regularity as for the diffusion case, at least when the jump satisfies certain integrability conditions.
This paper considers an optimal impulse control problem of dynamical systems generated by a flow. The performance criteria are total costs over the infinite time horizon. Apart from the main performance to be minimized, there are multiple constraints on performance functionals of a similar type. Under a natural set of compactness-continuity conditions on the system primitives, we establish a linear programming approach, and prove the existence of a stationary optimal control strategy out of a more general class of randomized strategies. This is done by making use of the tools from Markov decis…
This paper focuses on zero-sum stochastic differential games in the framework of forward-backward stochastic differential equations on a finite time horizon with both players adopting impulse controls. By means of BSDE methods, in particular that of the notion from Peng's stochastic \textit{% backward semigroups}, we prove a dynamic programming principle for both the upper and the lower value functions of the game. The upper and the lower value functions are then shown to be the unique viscosity solutions of the Hamilton-Jacobi-Bellman-Isaacs equations with a double-obstacle. As a consequence,…
We consider an optimal stochastic impulse control problem over an infinite time horizon motivated by a model of irreversible investment choices with fixed adjustment costs. By employing techniques of viscosity solutions and relying on semiconvexity arguments, we prove that the value function is a classical solution to the associated quasi-variational inequality. This enables us to characterize the structure of the continuation and action regions and construct an optimal control. Finally, we focus on the linear case, discussing, by a numerical analysis, the sensitivity of the solution with resp…
A two-person zero-sum infinite dimensional differential game of infinite duration with discounted payoff involving hybrid controls is studied. The minimizing player is allowed to take continuous, switching and impulse controls whereas the maximizing player is allowed to take continuous and switching controls. By taking strategies in the sense of Elliott--Kalton, we prove the existence of value and characterize it as the unique viscosity solution of the associated system of quasi-variational inequalities.
We consider an optimal switching problem where the terminal reward depends on the entire control trajectory. We show existence of an optimal control by applying a probabilistic technique based on the concept of Snell envelopes. We then apply this result to solve an impulse control problem for stochastic delay differential equations driven by a Brownian motion and an independent compound Poisson process. Furthermore, we show that the studied problem arises naturally when maximizing the revenue from operation of a group of hydro-power plants with hydrological coupling.
A non-autonomous evolution semi-linear differential system under non-instantaneous impulses, delays, and perturbed by non-local conditions is studied. Its piece-wise continuous solutions belong to a finite-dimensional Banach space. The existence and uniqueness of solutions on the interval $[-r,τ]$ are obtained by applying Karakostas fixed-point theorem. Further results concerning solution prolongation are developed. An example is presented, and several remarks on the infinite-dimensional case are included.
We prove a maximum principle for the problem of optimal control for a fractional diffusion with infinite horizon. Further, we show existence of fractional backward stochastic differential equations on infinite horizon. We illustrate our findings with an example.
We prove maximum principles for the problem of optimal control for a jump diffusion with infinite horizon and partial information. The results are applied to partial information optimal consumption and portfolio problems in infinite horizon.
This paper proposes penalty schemes for a class of weakly coupled systems of Hamilton-Jacobi-Bellman quasi-variational inequalities (HJBQVIs) arising from stochastic hybrid control problems of regime-switching models with both continuous and impulse controls. We show that the solutions of the penalized equations converge monotonically to those of the HJBQVIs. We further establish that the schemes are half-order accurate for HJBQVIs with Lipschitz coefficients, and first-order accurate for equations with more regular coefficients. Moreover, we construct the action regions and optimal impulse co…
We establish sufficient conditions for the existence and uniqueness of different types of delayed BSDEs in finite time horizon. We consider then infinite horizon, replacing the terminal value condition in the finite horizon case with a condition of strong decay at infinity.
In an infinite-horizon optimal control problem the stability of the adjoint variable implying the vanishing of the adjoint variable at infinity along optimal solution.
General theorems for existence and uniqueness of viscosity solutions for Hamilton-Jacobi-Bellman quasi-variational inequalities (HJBQVI) with integral term are established. Such nonlinear partial integro-differential equations (PIDE) arise in the study of combined impulse and stochastic control for jump-diffusion processes. The HJBQVI consists of an HJB part (for stochastic control) combined with a nonlocal impulse intervention term. Existence results are proved via stochastic means, whereas our uniqueness (comparison) results adapt techniques from viscosity solution theory. This paper is to o…
The concept of turnpike connects the solution of long but finite time horizon optimal control problems with steady state optimal controls. A key ingredient of the analysis of the turnpike is the linear quadratic regulator problem and the convergence of the solution of the associated differential Riccati equation as the terminal time approaches infinity. This convergence has been investigated in linear systems theory in the 1980s. We extend classical system theoretic results for the investigation of turnpike properties of standard state space systems and descriptor systems. We present condition…
This paper is concerned with a discounted optimal control problem of partially observed forward-backward stochastic systems with jumps on infinite horizon. The control domain is convex and a kind of infinite horizon observation equation is introduced. The uniquely solvability of infinite horizon forward (backward) stochastic differential equation with jumps is obtained and more extended analysis, especially for the backward case, is made. Some new estimates are first given and proved for the critical variational inequality. Then an ergodic maximum principle is obtained by introducing some infi…
We study so{ù}e infinite-horizon optimization problems on spaces of periodic functions for non periodic Lagrangians. The main strategy relies on the reduction to finite horizon thanks in the introduction of an avering operator.We then provide existence results and necessary optimality conditions in which the corresponding averaged Lagrangian appears.
We consider a problem of optimal control of an infinite horizon system governed by forward-backward stochastic differential equations with delay. Sufficient and necessary maximum principles for optimal control under partial information in infinite horizon are derived. We illustrate our results by an application to a problem of optimal consumption with respect to recursive utility from a cash flow with delay.
The paper addresses an existence problem for infinite horizon optimal control when the system under control is exponentially stabilizable or stable. Classes of nonlinear control systems for which infinite horizon optimal controls exist are identified in terms of stability, stabilizability, detectability and growth conditions. The result then applies to estimate the existence region of stable manifolds in the associated Hamiltonian systems. Applications of the results also include the analysis for turnpike property in nonlinear finite horizon optimal control problems by a geometric approach.
The extension of a conflict control problem with infinite horizon is constructed. This extension is the projective limit of restricted games. Relations between "sensitivity to target set" and the existence of the optimal control are studied. Special attention is paid to the pursuit-evasion game with "joint control/relaxed control".
Infinite horizon optimization problems accompany two perplexities. First, the infinite series of utility sequences may diverge. Second, boundary conditions at the infinite terminal time may not be rigorously expressed. In this paper, we show that under two fairly general conditions, the limit of the solution to the undiscounted finite horizon problem is optimal among feasible paths for the undiscounted infinite horizon problem, in the sense of the overtaking criterion. Applied to a simple Ramsey model, we show that the derived path contains intriguing properties. We also comprehend the legitim…
In the infinite-horizon and discrete-time framework we establish maximum principles of Pontryagin under assumptions which weaker than these ones of existing results. We avoid several assumptions of continuity and of Fr{é}chet-differentiability and of linear independence. MSC 2010: 49J21, 65K05, 39A99.
This technical report is an accompaniment to the paper "Differentially Private LQ Control" that is currently under review. This technical report provides a complete derivation of the infinite horizon discrete-time linear quadratic Gaussian tracking controller, which we were unable to find in the existing literature.
We prove a maximum principle of optimal control of stochastic delay equations on infinite horizon. We establish first and second sufficient stochastic maximum principles as well as necessary conditions for that problem. We illustrate our results by an application to the optimal consumption rate from an economic quantity.
This paper is devoted to a study of infinite horizon optimal control problems with time discounting and time averaging criteria in discrete time. We establish that these problems are related to certain infinite-dimensional linear programming (IDLP) problems. We also establish asymptotic relationships between the optimal values of problems with time discounting and long-run average criteria.
This note re-visits the rolling-horizon control approach to the problem of a Markov decision process (MDP) with infinite-horizon discounted expected reward criterion. Distinguished from the classical value-iteration approach, we develop an asynchronous on-line algorithm based on policy iteration integrated with a multi-policy improvement method of policy switching. A sequence of monotonically improving solutions to the forecast-horizon sub-MDP is generated by updating the current solution only at the currently visited state, building in effect a rolling-horizon control policy for the MDP over …
This paper studies the infinite-horizon sensor scheduling problem for linear Gaussian processes with linear measurement functions. Several important properties of the optimal infinite-horizon schedules are derived. In particular, it is proved that under some mild conditions, both the optimal infinite-horizon average-per-stage cost and the corresponding optimal sensor schedules are independent of the covariance matrix of the initial state. It is also proved that the optimal estimation cost can be approximated arbitrarily closely by a periodic schedule with a finite period. Moreover, it is shown…
This paper is concerned with a linear quadratic stochastic two-person zero-sum differential game with constant coefficients in an infinite time horizon. Open-loop and closed-loop saddle points are introduced. The existence of closed-loop saddle points is characterized by the solvability of an algebraic Riccati equation with a certain stabilizing condition. A crucial result makes our approach work is the unique solvability of a class of linear backward stochastic differential equations in an infinite horizon.
We investigate variational problems with recursive integral functionals governed by infinite-dimensional differential inclusions with an infinite horizon and present an existence result in the setting of nonreflexive Banach spaces. We find an optimal solution in a Sobolev space taking values in a Banach space under the Cesari type condition. We also investigate sufficient conditions for the existence of solutions to the initial value problem for the differential inclusion.
It has been recently established that a deterministic infinite horizon discounted optimal control problem in discrete time is closely related to a certain infinite dimensional linear programming problem and its dual. In the present paper, we use these results to establish necessary and sufficient optimality conditions for this optimal control problem and apply them to construct a near optimal control.
We study a zero-sum stochastic differential switching game in infinite horizon. We prove the existence of the value of the game and characterize it as the unique viscosity solution of the associated system of quasi-variational inequalities with bilateral obstacles. We also obtain a verification theorem which provides an optimal strategy of the game. Finally, some numerical examples with two regimes are given.
We consider problems of the calculus of variations on unbounded time scales. We prove the validity of the Euler-Lagrange equation on time scales for infinite horizon problems, and a new transversality condition.
In this paper we give feedback laws for a class of parametrized infinite horizon control problems under state constraints. We provide a two-player representation assuming time dependent dynamics and Lagrangian and the set constraints merely compact. Using viability results recently investigated for infinite horizon settings in presence of state constraints, we extend some known results for the linear-quadratic regulator problem to control problems with nonlinear dynamics in the state and affine in the control. Feedback laws are obtained under suitable controllability assumptions.
Continuing the first part of the paper, we consider scalar decentralized average-cost infinite-horizon LQG problems with two controllers. This paper focuses on the slow dynamics case when the eigenvalue of the system is small and prove that the single-controller optimal strategies ---linear strategies--- are constant ratio optimal among all distributed control strategies.
This paper concerns a class of infinite horizon optimal control problems with state constraints. By extending the needle variation method to the infinite horizon case we obtain a complete set of necessary optimality conditions for a strong local minimizer in form of the Pontryagin maximum principle and the validity of several transversality conditions at infinity. To this purpose, we develop an approach in the space of continuous functions converging at infinity. The considerations including new results on optimality and transversality conditions, on Banach spaces of continuous functions, on t…
