We give an effective sufficient condition for a variational problem with infinite horizon on a compact Riemannian manifold M to admit a smooth optimal synthesis, i. e. a smooth dynamical system on M whose positive semi-trajectories are solutions to the problem. To realize the synthesis we construct a well-projected to M invariant Lagrange submanifold of the extremals' flow in the cotangent bundle T*M. The construction uses the curvature of the flow in the cotangent bundle and some ideas of hyperbolic dynamics.
We investigate variational problems with recursive integral functionals governed by infinite-dimensional differential inclusions with an infinite horizon and present an existence result in the setting of nonreflexive Banach spaces. We find an optimal solution in a Sobolev space taking values in a Banach space under the Cesari type condition. We also investigate sufficient conditions for the existence of solutions to the initial value problem for the differential inclusion.
We consider problems of the calculus of variations on unbounded time scales. We prove the validity of the Euler-Lagrange equation on time scales for infinite horizon problems, and a new transversality condition.
We investigate the value function of an infinite horizon variational problem in the infinite-dimensional setting. Firstly, we provide an upper estimate of its Dini--Hadamard subdifferential in terms of the Clarke subdifferential of the Lipschitz continuous integrand and the Clarke normal cone to the graph of the set-valued mapping describing dynamics. Secondly, we derive a necessary condition for optimality in the form of an adjoint inclusion that grasps a connection between the Euler--Lagrange condition and the maximum principle.
This paper is concerned with dynamic user equilibrium (DUE) with elastic travel demand (E-DUE). We present and prove a variational inequality (VI) formulation of E-DUE using measure-theoretic argument. Moreover, existence of the E-DUE is formally established with a version of Brouwer's fixed point theorem in a properly defined Hilbert space. The existence proof requires the effective delay operator to be continuous, a regularity condition also needed to ensure the existence of DUE with fixed demand (Han et al., 2013c). Our proof does not invoke the a priori upper bound of the departure rates (…
We prove Euler-Lagrange type equations and transversality conditions for generalized infinite horizon problems of the calculus of variations on time scales. Here the Lagrangian depends on the independent variable, an unknown function and its nabla derivative, as well as a nabla indefinite integral that depends on the unknown function.
We prove a maximum principle for the problem of optimal control for a fractional diffusion with infinite horizon. Further, we show existence of fractional backward stochastic differential equations on infinite horizon. We illustrate our findings with an example.
We prove maximum principles for the problem of optimal control for a jump diffusion with infinite horizon and partial information. The results are applied to partial information optimal consumption and portfolio problems in infinite horizon.
We obtain Euler-Lagrange and transversality optimality conditions for higher-order infinite horizon variational problems on a time scale. The new necessary optimality conditions improve the classical results both in the continuous and discrete settings: our results seem new and interesting even in the particular cases when the time scale is the set of real numbers or the set of integers.
We establish sufficient conditions for the existence and uniqueness of different types of delayed BSDEs in finite time horizon. We consider then infinite horizon, replacing the terminal value condition in the finite horizon case with a condition of strong decay at infinity.
In an infinite-horizon optimal control problem the stability of the adjoint variable implying the vanishing of the adjoint variable at infinity along optimal solution.
This paper develops a variational inference framework for control of infinite dimensional stochastic systems. We employ a measure theoretic approach which relies on the generalization of Girsanov's theorem, as well as the relation between relative entropy and free energy. The derived control scheme is applicable to a large class of stochastic, infinite dimensional systems, and can be used for trajectory optimization and model predictive control. Our work opens up new research avenues at the intersection of stochastic control, inference and information theory for dynamical systems described by …
This paper is concerned with a discounted optimal control problem of partially observed forward-backward stochastic systems with jumps on infinite horizon. The control domain is convex and a kind of infinite horizon observation equation is introduced. The uniquely solvability of infinite horizon forward (backward) stochastic differential equation with jumps is obtained and more extended analysis, especially for the backward case, is made. Some new estimates are first given and proved for the critical variational inequality. Then an ergodic maximum principle is obtained by introducing some infi…
We study so{ù}e infinite-horizon optimization problems on spaces of periodic functions for non periodic Lagrangians. The main strategy relies on the reduction to finite horizon thanks in the introduction of an avering operator.We then provide existence results and necessary optimality conditions in which the corresponding averaged Lagrangian appears.
We consider a problem of optimal control of an infinite horizon system governed by forward-backward stochastic differential equations with delay. Sufficient and necessary maximum principles for optimal control under partial information in infinite horizon are derived. We illustrate our results by an application to a problem of optimal consumption with respect to recursive utility from a cash flow with delay.
The paper addresses an existence problem for infinite horizon optimal control when the system under control is exponentially stabilizable or stable. Classes of nonlinear control systems for which infinite horizon optimal controls exist are identified in terms of stability, stabilizability, detectability and growth conditions. The result then applies to estimate the existence region of stable manifolds in the associated Hamiltonian systems. Applications of the results also include the analysis for turnpike property in nonlinear finite horizon optimal control problems by a geometric approach.
The extension of a conflict control problem with infinite horizon is constructed. This extension is the projective limit of restricted games. Relations between "sensitivity to target set" and the existence of the optimal control are studied. Special attention is paid to the pursuit-evasion game with "joint control/relaxed control".
In the infinite-horizon and discrete-time framework we establish maximum principles of Pontryagin under assumptions which weaker than these ones of existing results. We avoid several assumptions of continuity and of Fr{é}chet-differentiability and of linear independence. MSC 2010: 49J21, 65K05, 39A99.
We consider impulse control of stochastic functional differential equations (SFDEs) driven by Lévy processes under an additional $L^p$-Lipschitz condition on the coefficients. Our results, which are first derived for a general stochastic optimization problem over infinite horizon impulse controls and then applied to the case of a controlled SFDE, apply to the infinite horizon as well as the random horizon settings. The methodology employed to show existence of optimal controls is a probabilistic one based on the concept of Snell envelopes.
This technical report is an accompaniment to the paper "Differentially Private LQ Control" that is currently under review. This technical report provides a complete derivation of the infinite horizon discrete-time linear quadratic Gaussian tracking controller, which we were unable to find in the existing literature.
We prove a maximum principle of optimal control of stochastic delay equations on infinite horizon. We establish first and second sufficient stochastic maximum principles as well as necessary conditions for that problem. We illustrate our results by an application to the optimal consumption rate from an economic quantity.
This paper is devoted to a study of infinite horizon optimal control problems with time discounting and time averaging criteria in discrete time. We establish that these problems are related to certain infinite-dimensional linear programming (IDLP) problems. We also establish asymptotic relationships between the optimal values of problems with time discounting and long-run average criteria.
This note re-visits the rolling-horizon control approach to the problem of a Markov decision process (MDP) with infinite-horizon discounted expected reward criterion. Distinguished from the classical value-iteration approach, we develop an asynchronous on-line algorithm based on policy iteration integrated with a multi-policy improvement method of policy switching. A sequence of monotonically improving solutions to the forecast-horizon sub-MDP is generated by updating the current solution only at the currently visited state, building in effect a rolling-horizon control policy for the MDP over …
This paper studies the infinite-horizon sensor scheduling problem for linear Gaussian processes with linear measurement functions. Several important properties of the optimal infinite-horizon schedules are derived. In particular, it is proved that under some mild conditions, both the optimal infinite-horizon average-per-stage cost and the corresponding optimal sensor schedules are independent of the covariance matrix of the initial state. It is also proved that the optimal estimation cost can be approximated arbitrarily closely by a periodic schedule with a finite period. Moreover, it is shown…
This paper is concerned with a linear quadratic stochastic two-person zero-sum differential game with constant coefficients in an infinite time horizon. Open-loop and closed-loop saddle points are introduced. The existence of closed-loop saddle points is characterized by the solvability of an algebraic Riccati equation with a certain stabilizing condition. A crucial result makes our approach work is the unique solvability of a class of linear backward stochastic differential equations in an infinite horizon.
It has been recently established that a deterministic infinite horizon discounted optimal control problem in discrete time is closely related to a certain infinite dimensional linear programming problem and its dual. In the present paper, we use these results to establish necessary and sufficient optimality conditions for this optimal control problem and apply them to construct a near optimal control.
We study a zero-sum stochastic differential switching game in infinite horizon. We prove the existence of the value of the game and characterize it as the unique viscosity solution of the associated system of quasi-variational inequalities with bilateral obstacles. We also obtain a verification theorem which provides an optimal strategy of the game. Finally, some numerical examples with two regimes are given.
In this paper we give feedback laws for a class of parametrized infinite horizon control problems under state constraints. We provide a two-player representation assuming time dependent dynamics and Lagrangian and the set constraints merely compact. Using viability results recently investigated for infinite horizon settings in presence of state constraints, we extend some known results for the linear-quadratic regulator problem to control problems with nonlinear dynamics in the state and affine in the control. Feedback laws are obtained under suitable controllability assumptions.
Continuing the first part of the paper, we consider scalar decentralized average-cost infinite-horizon LQG problems with two controllers. This paper focuses on the slow dynamics case when the eigenvalue of the system is small and prove that the single-controller optimal strategies ---linear strategies--- are constant ratio optimal among all distributed control strategies.
Infinite horizon backward stochastic Volterra integral equations (BSVIEs for short) are investigated. We prove the existence and uniqueness of the adapted M-solution in a weighted $L^2$-space. Furthermore, we extend some important known results for finite horizon BSVIEs to the infinite horizon setting. We provide a variation of constant formula for a class of infinite horizon linear BSVIEs and prove a duality principle between a linear (forward) stochastic Volterra integral equation (SVIE for short) and an infinite horizon linear BSVIE in a weighted $L^2$-space. As an application, we investiga…
This paper concerns a class of infinite horizon optimal control problems with state constraints. By extending the needle variation method to the infinite horizon case we obtain a complete set of necessary optimality conditions for a strong local minimizer in form of the Pontryagin maximum principle and the validity of several transversality conditions at infinity. To this purpose, we develop an approach in the space of continuous functions converging at infinity. The considerations including new results on optimality and transversality conditions, on Banach spaces of continuous functions, on t…
Our purpose of this paper is to study stochastic control problem for systems driven by mean-field stochastic differential equations with elephant memory, in the sense that the system (like the elephants) never forgets its history. We study both the finite horizon case and the infinite time horizon case. - In the finite horizon case, results about existence and uniqueness of solutions of such a system are given. Moreover, we prove sufficient as well as necessary stochastic maximum principles for the optimal control of such systems. We apply our results to solve a mean-field linear quadratic con…
In this paper we deal with infinite horizon optimal control problems. Basing on weak variations in an extremal problem in weighted function spaces we prove necessary conditions in form of the adjoint equation and a variational inequality. The obtained optimality conditions imply several transversality conditions and the validity of Mangasarian type sufficiency conditions. Arising difficulties in the presence of state constraints are discussed.
Infinite horizon open loop optimal control problems for semilinear parabolic equations are investigated. The controls are subject to a cost-functional which promotes sparsity in time. The focus is put on deriving first order optimality conditions. This is achieved without relying on a well-defined control-to-state mapping in a neighborhood of minimizers. The technique of proof is based on the approximation of the original problem by a family of finite horizon problems. The optimality conditions allow to deduce sparsity properties of the optimal controls in time.
A finite horizon optimal stopping problem for an infinite dimensional diffusion $X$ is analyzed by means of variational techniques. The diffusion is driven by a SDE on a Hilbert space $\mathcal{H}$ with a non-linear diffusion coefficient $σ(X)$ and a generic unbounded operator $A$ in the drift term. When the gain function $Θ$ is time-dependent and fulfils mild regularity assumptions, the value function $\mathcal{U}$ of the optimal stopping problem is shown to solve an infinite-dimensional, parabolic, degenerate variational inequality on an unbounded domain. Once the coefficient $σ(X)$ is speci…
In this paper we summarize our results in infinite horizon optimal control. We present optimality conditions for weak local minimizer in the framework of weighted functions. Moreover we formulate the Pontryagin Maximum Principle for strong local minimizer for trajectories converging at infinity. The considered problems, the requirements and the results depend on the choice of the function space.
This paper is concerned with a stochastic linear-quadratic (LQ) optimal control problem on infinite time horizon, with regime switching, random coefficients, and cone control constraint. To tackle the problem, two new extended stochastic Riccati equations (ESREs) on infinite time horizon are introduced. The existence of the nonnegative solutions, in both standard and singular cases, is proved through a sequence of ESREs on finite time horizon. Based on this result and some approximation techniques, we obtain the optimal state feedback control and optimal value for the stochastic LQ problem exp…
A class of infinite horizon optimal control problems involving $L^p$-type cost functionals with $0<p\leq 1$ is discussed. The existence of optimal controls is studied for both the convex case with $p=1$ and the nonconvex case with $0<p<1$, and the sparsity structure of the optimal controls promoted by the $L^p$-type penalties is analyzed. A dynamic programming approach is proposed to numerically approximate the corresponding sparse optimal controllers.
In this paper, we show existence and uniqueness of solutions of the infinite horizon McKean-Vlasov FBSDEs using two different methods, which lead to two different sets of assumptions. We use these results to solve the infinite horizon mean field type control problems and mean field games.
In this paper, we consider how to construct the optimal solutions for a general discrete time infinite horizon optimal control problem. We establish necessary and sufficient conditions for optimality in the sense of a modified optimality criterion. We also consider how transversality conditions are related to steady states. The results are applied to two examples to demonstrate how the new transversality conditions derived in this paper differ from those given in the previous literature.
The minimization of energy-like cost functionals is addressed in the context of optimal control problems. For a general class of dynamical systems, with possibly unstable and nonlinear free dynamics, it is shown that a sequence of solutions of finite time-horizon optimal control problems approximates a solution of the analog infinite time-horizon problem. The latter solution and corresponding optimal cost value function are not assumed to be known a-priori. Numerical simulations are presented validating the theoretical findings for several examples, including systems governed by both ordinary …
New form of sufficient optimality condition is obtained in comparison with the Mangasarian sufficiency theorem. Both finite and infinite values of objective functional are allowed since concepts of overtaking and weakly overtaking optimality are implied. Examples, where new conditions can be applied are presented. The conditions are shown to be both necessary and sufficient, when Hamiltonian is linear with respect to state and control.
Necessary optimality conditions in the form of the maximum principle for control problems with infinite time horizon are considered. Both finite and infinite values of objective functional are allowed since the concept of overtaking or weakly overtaking optimality is used. New form of optimality condition is obtained and compared with the transversality conditions usually used in the literature. The examples, where these transversality conditions may fail while the new condition holds are presented. For Ramsey problem of capital accumulation a simple form of necessary optimality conditions is …
