We consider an optimal control problem for a system governed by a Volterra integral equation with impulsive terms. The impulses act on both the state and the control; the control consists of switchings at discrete times. The cost functional includes both, an integrated cost rate (continuous part) and switching costs at the discrete impulse times (discrete part). We prove necessary optimality conditions of a form analogous to a discrete maximum principle. For the particular case of a system governed by impulsive ordinary differential equations, we obtain an impulsive maximum principle as a special case of the necessary optimality conditions for impulsive Volterra equations.
We obtain necessary conditions of optimality for impulsive Volterra integral equations with switching and impulsive controls, with variable impulse time-instants. The present work continues and complements our previous work on impulsive Volterra control with fixed impulse times.
We present and analyze a new method for solving optimal control problems for Volterra integral equations, based on approximating the controlled Volterra integral equations by a sequence of systems of controlled ordinary differential equations. The resulting approximating problems can then be solved by dynamic programming methods for ODE controlled systems. Other, straightforward versions of dynamic programming, are not applicable to Volterra integral equations. We also derive the connection between our version of dynamic programming and the Hamiltonian equations for Volterra controlled systems…
We study the problem of optimal control of a coupled system of forward-backward stochastic Volterra equations. We use Hida-Malliavin calculus to prove a sufficient and a necessary maximum principle for the optimal control of such systems. Existence and uniqueness of backward stochastic Volterra integral equations are proved. As an application of our methods, we solve a recursive utility optimisation problem in a financial model with memory.
Optimal control problems of forward-backward stochastic Volterra integral equations (FBSVIEs in short) are formulated and studied. A general duality principle is established for linear backward stochastic integral equation and linear stochastic Fredholm-Volterra integral equation with mean-field. With the help of such a duality principle, together with some other new delicate and subtle skills, Pontryagin type maximum principles are proved for two optimal control problems of FBSVIEs.
We study optimal control of stochastic Volterra integral equations (SVIE) with jumps by using Hida-Malliavin calculus. - We give conditions under which there exists unique solutions of such equations. - Then we prove both a sufficient maximum principle (a verification theorem) and a necessary maximum principle via Hida-Malliavin calculus. - As an application we solve a problem of optimal consumption from a cash flow modelled by an SVIE.
We study the problem of optimal inside control of a stochastic Volterra equation driven by a Brownian motion and a Poisson random measure. We prove a sufficient and a necessary maximum principle for the optimal control when the trader has only partial information available to her decisions and on the other hand, may have some inside information about the future of the system. The results are applied to the problem of finding the optimal insider portfolio in a financial market where the risky asset price is given by a stochastic Volterra equation.
Optimal control problems of forward-backward stochastic Volterra integral equations (FBSVIEs, in short) with closed control regions are formulated and studied. Instead of using spike variation method as one may imagine, here we turn to treat the non-convexity of the control regions by borrowing some tools in set-valued analysis and adapting them into our stochastic control systems. A duality principle between linear backward stochastic Volterra integral equations and linear stochastic Fredholm-Volterra integral equations with conditional expectation are derived, which extends and improves the …
This paper is concerned with a linear quadratic optimal control for a class of singular Volterra integral equations. Under proper convexity conditions, optimal control uniquely exists, and it could be characterized via Frechet derivative of the quadratic functional in a Hilbert space or via maximum principle type necessary conditions. However, these (equivalent) characterizations have a shortcoming that the current value of the optimal control depends on the future values of the optimal state. Practically, this is not feasible. The main purpose of this paper is to obtain a causal state feedbac…
We consider the optimal control problem of minimizing some quadratic functional over all possible solutions of an internally controlled multi-dimensional heat equation with a periodic terminal state constraint. This problem has a unique optimal solution, which can be characterized by an optimality system derived from the Pontryagin maximum principle. We define two approximations of this optimal control problem. The first one is an impulse approximation, and consists of considering a system of linear heat equations with impulse control. The second one is obtained by the sample-and-hold procedur…
Using the tools of the Markov Decision Processes, we justify the dynamic programming approach to the optimal impulse control of deterministic dynamical systems. We prove the equivalence of the integral and differential forms of the optimality equation. The theory is illustrated by an example from mathematical epidemiology. The developed methods can be also useful for the study of piecewise deterministic Markov processes.
Optimal control problems of forward stochastic Volterra integral equations (SVIEs) are formulated and studied. When control region is arbitrary subset of Euclidean space and control enters into the diffusion, necessary conditions of Pontryagin's type for optimal controls are established via spike variation. Our conclusions naturally cover the analogue of stochastic differential equations (SDEs), and our developed methodology drops the reliance on Itô formula and second-order adjoint equations. Some new features, that are concealed in the SDEs framework, are revealed in our situation. For examp…
We discuss the problem of optimal impulse control representing the preventive maintenance of a simple reparable system. The system model is governed by coupled transport and integro-differential equations in a nonreflexive Banach space. The objective of this paper is to construct nonnegative impulse control inputs at given system running times that minimize the probability of the system in failure mode. To guarantee the nonnegativity of the controlled system, we consider the control inputs to depend on the system state. This essentially leads to a bilinear control problem. We first present a r…
A linear-quadratic optimal control problem for a forward stochastic Volterra integral equation (FSVIE, for short) is considered. Under the usual convexity conditions, open-loop optimal control exists, which can be characterized by the optimality system, a coupled system of an FSVIE and a Type-II backward SVIE (BSVIE, for short). To obtain a causal state feedback representation for the open-loop optimal control, a path-dependent Riccati equation for an operator-valued function is introduced, via which the optimality system can be decoupled. In the process of decoupling, a Type-III BSVIE is intr…
In this paper, we study the optimal control problem with terminal and inequality state constraints for state equations described by Volterra integral equations having singular and nonsingular kernels. The singular kernel introduces abnormal behavior of the state trajectory with respect to the parameter of $α\in (0,1)$. Our state equation is able to cover various state dynamics such as any types of Volterra integral equations with nonsingular kernels only, fractional differential equations (in the sense of Riemann-Liouville or Caputo), and ordinary differential state equations. We obtain the we…
We study linear-quadratic optimal control problems for Voterra systems, and problems that are linear-quadratic in the control but generally nonlinear in the state. In the case of linear-quadratic Volterra control, we obtain sharp necessary and sufficient conditions for optimality. For problems that are linear-quadratic in the control only, we obtain a novel form of necessary conditions in the form of double Volterra equation; we prove the solvability of such equations.
The aim of the paper is to study an optimal control problem on infinite horizon for an infinite dimensional integro-differential equation with completely monotone kernelskernels, where we assume that the noise enters the system when we introduce a control. We start by reformulating the state equation into a semilinear evolution equation which can be treated by semigroup methods. The application to optimal control provide other interesting result and require a precise descriprion of the properties of the generated semigroup. The main tools consist in studying the differentiability of the forwar…
We analyze an optimal control problem for systems of integral equations of Volterra type with two independent variables. These systems generalize both, the hyperbolic control problems for systems of Goursat-Darboux type, and the optimal control of ordinary (i.e. with one independent variable) Volterra integral equations. We prove extremal principles akin to Pontryagin's maximum principle.
This paper considers an optimal impulse control problem of dynamical systems generated by a flow. The performance criteria are total costs over the infinite time horizon. Apart from the main performance to be minimized, there are multiple constraints on performance functionals of a similar type. Under a natural set of compactness-continuity conditions on the system primitives, we establish a linear programming approach, and prove the existence of a stationary optimal control strategy out of a more general class of randomized strategies. This is done by making use of the tools from Markov decis…
We analyze optimal control problems for multiple Fredholm and Volterra integral equations. These are non Pontryaginian optimal control problems, i.e. an extremum principle of Pontryagin type does not hold. We obtain first order necessary conditions for optimality, and second order necessary and sufficient conditions. We illustrate with applications to first order and second order Volterra bilinear control problems.
Deterministic optimal impulse control problem with terminal state constraint is considered. Due to the appearance of the terminal state constraint, the value function might be discontinuous in general. The main contribution of this paper is the introduction of an intrinsic condition under which the value function is continuous. Then by a Bellman dynamic programming method, the corresponding Hamilton-Jacobi-Bellman type quasi-variational inequality (QVI, for short) is derived for which the value function is a viscosity solution. The issue of whether the value function is characterized as the un…
We formulate and analyze a new method for solving optimal control problems for systems governed by Volterra integral equations. Our method utilizes discretization of the original Volterra controlled system and a novel type of dynamic programming jn which the Hamilton-Jacobi function is parametrized by the control function (rather than the state, as in the case of ordinary dynamic programming). We also derive estimates for the computational cost of our method.
The paper is concerned with a kind of minimal time control problem for the heat equation with impulse controls. The purpose of such a problem is to find an optimal impulse control (among certain control constraint set) steering the solution of the heat equation from a given initial state to a given target set as soon as possible. We will first study the existence and uniqueness of optimal solution for this problem. In the formulation of this problem, there are two parameters: one is the upper bound of the control constraint and the other one is the moment of impulse time. Then, we will establi…
For a constrained optimal impulse control problem of an abstract dynamical system, we introduce the occupation measures along with aggregated occupation measures and present two associated linear programs. We prove that the two linear programs are equivalent under appropriate conditions, and each linear program gives rise to an optimal strategy in the original impulse control problem.
This paper establishes a verification theorem for impulse control problems involving conditional McKean-Vlasov jump diffusions. We obtain a Markovian system by combining the state equation of the problem with the stochastic Fokker-Planck equation for the conditional probability law of the state. We derive sufficient variational inequalities for a function to be the value function of the impulse control problem, and for an impulse control to be the optimal control. We illustrate our results by applying them to the study of an optimal stream of dividends under transaction costs. We obtain the so…
Backward doubly stochastic Volterra integral equations (BDSVIEs, for short) are introduced and studied systematically. Well-posedness of BDSVIEs in the sense of introduced M-solutions is established. A comparison theorem for BDSVIEs is proved. By virtue of the comparison theorem, we derive the existence of solutions for BDSVIEs with continuous coefficients. Furthermore, a duality principle between linear (forward) doubly stochastic Volterra integral equation (FDSVIE, for short) and BDSVIE is obtained. A Pontryagin type maximum principle is also established for an optimal control problem of FDS…
Spike variation technique plays a crucial role in deriving Pontryagin's type maximum principle of optimal controls for differential equations of several types, including ordinary differential equations (ODEs), partial differential equations (PDEs), and stochastic differentia equations (SDEs), when the control domains are not assumed to be convex. This technique also applies to (deterministic forward) Volterra intrgral equations (FVIEs). It is natural to expect that such a technique could be extended to the case of (forward) stochastic Volterra integral equations (FSVIEs). However, by mimicking…
This paper is concerned with a class of controlled singular Volterra integral equations, which could be used to describe problems involving memories. The well-known fractional order ordinary differential equations of the Riemann--Liouville or Caputo types are strictly special cases of the equations studied in this paper. Well-posedness and some regularity results in proper spaces are established for such kind of questions. For the associated optimal control problem, by using a Liapounoff's type theorem and the spike variation technique, we establish a Pontryagin's type maximum principle for op…
Families of Two-Impulse Optimal Rendezvous Transfers Between Elliptic Orbits
School of Aeronautics and Astronautics, Purdue University（普渡大学航空航天学院）；Department of Aerospace Engineering, Korea Advanced Institute of Science and Technology（韩国科学技术院航空航天工程系）
In this paper we consider impulse control of continuous time Markov processes with average cost per unit time functional. This problem is approximated using impulse control problems stopped at the first exit time from increasing sequence of open sets. We find solution to Bellman equation corresponding to the original problem and show that stopped impulse control problems approximate optimal value of the cost functional.
In this paper, we prove both necessary and sufficient maximum principles for infinite horizon discounted control problems of stochastic Volterra integral equations with finite delay and a convex control domain. The corresponding adjoint equation is a novel class of infinite horizon anticipated backward stochastic Volterra integral equations. Our results can be applied to discounted control problems of stochastic delay differential equations and fractional stochastic delay differential equations. As an example, we consider a stochastic linear-quadratic regulator problem for a delayed fractional…
In this paper, we consider joint drift rate control and impulse control for a stochastic inventory system under long-run average cost criterion. Assuming the inventory level must be nonnegative, we prove that a $\{(0,q^{\star},Q^{\star},S^{\star}),\{μ^{\star}(x): x\in[0, S^{\star}]\}\}$ policy is an optimal joint control policy, where the impulse control follows the control band policy $(0,q^{\star},Q^{\star},S^{\star})$, that brings the inventory level up to $q^{\star}$ once it drops to $0$ and brings it down to $Q^{\star}$ once it rises to $S^{\star}$, and the drift rate only depends on the …
This paper is devoted to the stochastic optimal control problems for systems governed by forward-backward stochastic Volterra integral equations (FBSVIEs, for short) with state constraints. Using Ekeland's variational principle, we obtain one kind of variational inequality. Then, by dual method, we derive a stochastic maximum principle which gives the necessary conditions for the optimal controls.
A method to compute optimal collision avoidance maneuvers for short-term encounters is presented. The maneuvers are modeled as multiple-impulses to handle impulsive cases and to approximate finite burn arcs associated either with short alert times or the use of low-thrust propulsion. The maneuver design is formulated as a sequence of convex optimization problems solved in polynomial time by state-of-the-art primal-dual interior-point algorithms. The proposed approach calculates optimal solutions without assumptions about the thrust arc structure and thrust direction. The execution time is frac…
Solutions of stochastic Volterra (integral) equations are not Markov processes, and therefore classical methods, like dynamic programming, cannot be used to study optimal control problems for such equations. However, we show that by using {\em Malliavin calculus} it is possible to formulate a modified functional type of {\em maximum principle} suitable for such systems. This principle also applies to situations where the controller has only partial information available to base her decisions upon. We present both a sufficient and a necessary maximum principle of this type, and then we use the …
This paper studies an optimal stochastic impulse control problem in a finite horizon with a decision lag, by which we mean that after an impulse is made, a fixed number units of time has to be elapsed before the next impulse is allowed to be made. The continuity of the value function is proved. A suitable version of dynamic programming principle is established, which takes into account the dependence of state process on the elapsed time. The corresponding Hamilton-Jacobi-Bellman (HJB) equation is derived, which exhibit some special feature of the problem. The value function of this optimal imp…
We consider an investment problem in which an investor performs capital injections to increase the liquidity of a firm for it to maximise profit from market operations. Each time the investor performs an injection, the investor incurs a fixed transaction cost. In addition to maximising their terminal reward, the investor seeks to minimise the risk of loss of their investment (from a possible firm ruin) by exiting the market at some point in time. We show that the problem can be reformulated in terms of a new stochastic differential game of control and stopping in which one of the players modif…
We consider impulse control of stochastic functional differential equations (SFDEs) driven by Lévy processes under an additional $L^p$-Lipschitz condition on the coefficients. Our results, which are first derived for a general stochastic optimization problem over infinite horizon impulse controls and then applied to the case of a controlled SFDE, apply to the infinite horizon as well as the random horizon settings. The methodology employed to show existence of optimal controls is a probabilistic one based on the concept of Snell envelopes.
In this paper we are concerned with a class of stochastic Volterra integro-differential problems with completely monotone kernels, where we assume that the noise enters the system when we introduce a control. We start by reformulating the state equation into a semilinear evolution equation which can be treated by semigroup methods. The application to optimal control provide other interesting result and require a precise descriprion of the properties of the generated semigroup. The first main result of the paper is the proof of existence and uniqueness of a mild solution for the corresponding H…
The purpose of this paper is to investigate the role that the continuous-time generalised Riccati equation plays within the context of singular linear-quadratic optimal control. This equation has been defined following the analogy with the discrete-time generalised Riccati equation, but, differently from the discrete case, to date the importance of this equation in the context of optimal control is yet to be understood. This note addresses this point. We show in particular that when the continuous-time generalised Riccati equation admits a symmetric solution, the corresponding linear-quadratic…
We consider stochastic impulse control problems where the process is driven by a general one-dimensional diffusion. We shall show a new mathematical characterization of the value function as a linear function in a certain transformed space. Our approach can relieve us from the burden of guessing and proving the optimal strategy, (2) present a simple method to find the value function and the corresponding control policies, and (3) handle systematically a broader class of reward and cost functions since the existence of the value function can be shown in much a simpler way.
We consider a class of stochastic impulse control problems of general stochastic processes i.e. not necessarily Markovian. Under fairly general conditions we establish existence of an optimal impulse control. We also prove existence of combined optimal stochastic and impulse control of a fairly general class of diffusions with random coefficients. Unlike, in the Markovian framework, we cannot apply quasi-variational inequalities techniques. We rather derive the main results using techniques involving reflected BSDEs and the Snell envelope.
An optimal control problem is considered for a stochastic differential equation with the cost functional determined by a backward stochastic Volterra integral equation (BSVIE, for short). This kind of cost functional can cover the general discounting (including exponential and non-exponential) situation with a recursive feature. It is known that such a problem is time-inconsistent in general. Therefore, instead of finding a global optimal control, we look for a time-consistent locally near optimal equilibrium strategy. With the idea of multi-person differential games, a family of approximate e…
