This paper concerns a class of infinite horizon optimal control problems with state constraints. By extending the needle variation method to the infinite horizon case we obtain a complete set of necessary optimality conditions for a strong local minimizer in form of the Pontryagin maximum principle and the validity of several transversality conditions at infinity. To this purpose, we develop an approach in the space of continuous functions converging at infinity. The considerations including new results on optimality and transversality conditions, on Banach spaces of continuous functions, on the needle variation method and on problems with state constraints on the unbounded time horizon.
In the infinite-horizon and discrete-time framework we establish maximum principles of Pontryagin under assumptions which weaker than these ones of existing results. We avoid several assumptions of continuity and of Fr{é}chet-differentiability and of linear independence. MSC 2010: 49J21, 65K05, 39A99.
In this work, we address some optimal control problems related to the evolution of two isothermal, incompressible, immisible fluids in a two dimensional bounded domain. A distributed optimal control problem is formulated as the minimization of a suitable cost functional subject to the controlled nonlocal Cahn-Hilliard-Navier-Stokes equations. We describe the first order necessary conditions of optimality via Pontryagin minimum principle and prove second order necessary and sufficient conditions of optimality for the problem.
The aim of this paper is to establish Pontryagin's principles in a dicrete-time infinite-horizon setting when the state variables and the control variables belong to infinite dimensional Banach spaces. In comparison with previous results on this question, we delete conditions of finiteness of codi-mension of subspaces. To realize this aim, the main idea is the introduction of new recursive assumptions and useful consequences of the Baire category theorem and of the Banach isomorphism theorem.
This article considers a discrete-time robust optimal control problem on matrix Lie groups. The underlying system is assumed to be perturbed by exogenous unmeasured bounded disturbances, and the control problem is posed as a min-max optimal control wherein the disturbance is the adversary and tries to maximise a cost that the control tries to minimise. Assuming the existence of a saddle point in the problem, we present a version of the Pontryagin maximum principle (PMP) that encapsulates first-order necessary conditions that the optimal control and disturbance trajectories must satisfy. This P…
Necessary conditions of optimality in the form of the Pontryagin Maximum Principle are derived for the Bolza-type discounted problem with free right end. The optimality is understood in the sense of the uniformly overtaking optimality. Such process is assumed to exist, and the corresponding payoff of the optimal process (expressed in the form of improper integral) is assumed to converge in the Riemann sense. No other assumptions on the asymptotic behaviour of trajectories or adjoint variables are required. In this paper, we prove that there exists a corresponding limiting solution of the Pontr…
Horizon-Uniform Sensitivity and Decay of Terminal Reward Perturbations in Discrete-Time Pontryagin Systems
School of Cyber Science and Engineering, Sichuan University, Chengdu, Sichuan, China（四川大学信息科学与工程学院，成都，四川，中国）；School of Mathematics, Sichuan University, Chengdu, Sichuan, China（四川大学数学学院，成都，四川，中国）
In this paper we summarize our results in infinite horizon optimal control. We present optimality conditions for weak local minimizer in the framework of weighted functions. Moreover we formulate the Pontryagin Maximum Principle for strong local minimizer for trajectories converging at infinity. The considered problems, the requirements and the results depend on the choice of the function space.
This paper introduces a framework for solving time-autonomous nonlinear infinite horizon optimal control problems, under the assumption that all minimizers satisfy Pontryagin's necessary optimality conditions. In detail, we use methods from the field of symplectic geometry to analyze the eigenvalues of a Koopman operator that lifts Pontryagin's differential equation into a suitably defined infinite dimensional symplectic space. This has the advantage that methods from the field of spectral analysis can be used to characterize globally optimal control laws. A numerical method for constructing o…
The main purpose of this paper is to give a solution to a long-standing unsolved problem in stochastic control theory, i.e., to establish the Pontryagin-type maximum principle for optimal controls of general infinite dimensional nonlinear stochastic evolution equations. Both drift and diffusion terms can contain the control variables, and the control domains are allowed to be nonconvex. The key to reach it is to provide a suitable formulation of operator-valued backward stochastic evolution equations (BSEEs for short), as well as a way to define their solutions. Besides, both vector-valued and…
In this paper we investigate necessary conditions of optimality for infinite-horizon optimal control problems with overtaking optimality as an optimality criterion. For the case of local Lipschitz continuity of the payoff function, we construct a boundary condition on the co-state arc that is necessary for the optimality. We also show that, under additional assumptions on the payoff function's asymptotic behavior, the Pontryagin Maximum Principle with this condition becomes a complete system of relations, and this boundary condition points out the unique co-state arc through a Cauchy-type form…
This paper investigates the necessary optimality conditions for uniformly overtaking optimal control on infinite horizon in the free end case. %with free right endpoint. In the papers of S.M.Aseev, A.V.Kryazhimskii, V.M.Veliov, K.O.Besov there was suggested the boundary condition for equations of the Pontryagin Maximum Principle. Each optimal process corresponds to a unique solution satisfying the boundary condition. Following A.Seierstad's idea, in this paper we prove a more general geometric variety of that boundary condition. We show that this condition is necessary for uniformly overtaking…
Reliable high-fidelity quantum state transformation has always been considered as an inseparable part of quantum information processing. In this regard, Pontryagin maximum principle has proved to play an important role to achieve the maximum fidelity in an optimum time or energy. Motivated by this, in this work, we formulate a control constrained optimal control problem where we aim to minimize time and also energy subjected to a quantum system satisfying the bilinear Schrodinger equation. We derive the first order optimality conditions through the application of Pontryagin Maximum (minimum) P…
We investigate conditions of optimality for an infinite horizon control problem and consider their correspondence with the value function. Assuming Lipschitz continuity of the value function, we prove that sensitivity relations plus the normal form version of the Pontryagin Maximum Principle is a necessary and sufficient condition for the optimality criteria that correspond to this value function. Different criteria of optimality under different asymptotic constraints may be used, including almost strong and classical optimality proposed by D.Bogusz. Special attention is devoted to weakly agre…
This paper studies a dynamic real-time optimization in the context of model-based time-optimal operation of batch processes under parametric model mismatch. In order to tackle the model-mismatch issue, a receding-horizon policy is usually followed with frequent re-optimization. The main problem addressed in this study is the high computational burden that is usually required by such schemes. We propose an approach that uses parameterized conditions of optimality in the adaptive predictive-control fashion. The uncertainty in the model predictions is treated explicitly using reachable sets that …
We consider the Lagrange problem of optimal control with unrestricted controls and address the question: under what conditions we can assure optimal controls are bounded? This question is related to the one of Lipschitzian regularity of optimal trajectories, and the answer to it is crucial for closing the gap between the conditions arising in the existence theory and necessary optimality conditions. Rewriting the Lagrange problem in a parametric form, we obtain a relation between the applicability conditions of the Pontryagin maximum principle to the later problem and the Lipschitzian regulari…
National Research Council（美国国家研究委员会）；Air Force Research Laboratory（美国空军研究实验室）；Texas A & M University（德州农工大学）；Infoscitex Corp.（Infoscitex公司）
The Dubins interval problem aims to find the shortest path of bounded curvature between two targets such that the departure angle from the first target and the arrival angle at the second target are constrained to two respective intervals. We propose a new and a simple algorithm to this problem based on the minimum principle of Pontryagin.
In this paper we consider the minimum time population transfer problem for a two level quantum system driven by {\em two} external fields with bounded amplitude. The controls are modeled as real functions and we do not use the Rotating Wave Approximation. After projection on the Bloch sphere, we tackle the time-optimal control problem with techniques of optimal synthesis on 2-D manifolds. Based on the Pontryagin Maximum Principle, we characterize a restricted set of candidate optimal trajectories. Properties on this set, crucial for complete optimal synthesis, are illustrated by numerical simu…
The paper concerns the study of the Pontryagin Maximum Principle for an infinite dimensional and infinite horizon boundary control problem for linear partial differential equations. The optimal control model has already been studied both in finite and infinite horizon with Dynamic Programming methods in a series of papers by the same author, or by Faggian and Gozzi. Necessary and sufficient optimality conditions for open loop controls are established. Moreover the co-state variable is shown to coincide with the spatial gradient of the value function evaluated along the trajectory of the system…
We provide a solution to the problem of receding horizon control for stochastic discrete-time systems with bounded control inputs and imperfect state measurements. For a suitable choice of control policies, we show that the finite-horizon optimization problem to be solved on-line is convex and successively feasible. Due to the inherent nonlinearity of the feedback loop, a slight extension of the Kalman filter is exploited to estimate the state optimally in mean-square sense. We show that the receding horizon implementation of the resulting control policies renders the state of the overall syst…
Infinite horizon backward stochastic Volterra integral equations (BSVIEs for short) are investigated. We prove the existence and uniqueness of the adapted M-solution in a weighted $L^2$-space. Furthermore, we extend some important known results for finite horizon BSVIEs to the infinite horizon setting. We provide a variation of constant formula for a class of infinite horizon linear BSVIEs and prove a duality principle between a linear (forward) stochastic Volterra integral equation (SVIE for short) and an infinite horizon linear BSVIE in a weighted $L^2$-space. As an application, we investiga…
A Pontryagin maximum principle for an optimal control problem in three dimensional linearized compressible viscous flows is established using the Ekeland variational principle. The controls are distributed over a bounded domain, while the state variables are subject to a set of constraints and governed by the linearized compressible Navier-Stokes equations. The maximum principle is of integral-type and obtained for minimizers of a tracking-type integral cost functional.
We use the continuation and bifurcation package pde2path to numerically analyze infinite time horizon optimal control problems for parabolic systems of PDEs. The basic idea is a two step approach to the canonical systems, derived from Pontryagin's maximum principle. First we find branches of steady or time-periodic states of the canonical systems, i.e., canonical steady states (CSS) respectively canonical periodic states (CPS), and then use these results to compute time-dependent canonical paths connecting to a CSS or a CPS with the so called saddle point property. This is a (high dimensional)…
This paper proves continuity of value functions in discounted periodic-review single-commodity total-cost inventory control problems with \revision{continuous inventory levels,} fixed ordering costs, possibly bounded inventory storage capacity, and possibly bounded order sizes for finite and infinite horizons. In each of these constrained models, the finite and infinite-horizon value functions are continuous, there exist deterministic Markov optimal finite-horizon policies, and there exist stationary deterministic Markov optimal infinite-horizon policies. For models with bounded inventory stor…
In this paper, we formulate a distributed optimal control problem related to the evolution of two isothermal, incompressible, immiscible fluids in a two dimensional bounded domain. The distributed optimal control problem is framed as the minimization of a suitable cost functional subject to the controlled nonlocal Cahn-Hilliard-Navier-Stokes equations. We describe the first order necessary conditions of optimality via Pontryagin's minimum principle and prove second order necessary and sufficient conditions of optimality for the problem.
We consider a simple control problem in which the underlying dynamics depend on a parameter that is unknown and must be learned. We exhibit a control strategy which is optimal to within a multiplicative constant. While most authors find strategies which are successful as the time horizon tends to infinity, our strategy achieves lowest expected cost up to a constant factor for a fixed time horizon.
We study a continuous-time, finite horizon, stochastic partially reversible investment problem for a firm producing a single good in a market with frictions. The production capacity is modeled as a one-dimensional, time-homogeneous, linear diffusion controlled by a bounded variation process which represents the cumulative investment-disinvestment strategy. We associate to the investment-disinvestment problem a zero-sum optimal stopping game and characterize its value function through a free-boundary problem with two moving boundaries. These are continuous, bounded and monotone curves that solv…
This paper is concerned with the problem of Model Predictive Control and Rolling Horizon Control of discrete-time systems subject to possibly unbounded random noise inputs, while satisfying hard bounds on the control inputs. We use a nonlinear feedback policy with respect to noise measurements and show that the resulting mathematical program has a tractable convex solution in both cases. Moreover, under the assumption that the zero-input and zero-noise system is asymptotically stable, we show that the variance of the state, under the resulting Model Predictive Control and Rolling Horizon Contr…
We design receding horizon control strategies for stochastic discrete-time linear systems with additive (possibly) unbounded disturbances, while obeying hard bounds on the control inputs. We pose the problem of selecting an appropriate optimal controller on vector spaces of functions and show that the resulting optimization problem has a tractable convex solution. Under the assumption that the zero-input and zero-noise system is asymptotically stable, we show that the variance of the state is bounded when enforcing hard bounds on the control inputs, for any receding horizon implementation. Thr…
We consider an optimization problem where the decision variable is a string of bounded length. For some time there has been an interest in bounding the performance of the greedy strategy for this problem. Here, we provide weakened sufficient conditions for the greedy strategy to be bounded by a factor of $(1-(1-1/K)^K)$, where $K$ is the optimization horizon length. Specifically, we introduce the notions of $K$-submodularity and $K$-GO-concavity, which together are sufficient for this bound to hold. By introducing a notion of \emph{curvature} $η\in(0,1]$, we prove an even tighter bound with th…
This paper studies an optimal control problem related to membrane filtration processes. A simple mathematical model of membrane fouling is used to capture the dynamic behavior of the filtration process which consists in the attachment of matter onto the membrane during the filtration period and the detachment of matter during the cleaning period. We consider the maximization of the net production of a membrane filtration system (i.e. the filtrate) over a finite time horizon, where control variable is the sequence of filtration/backwashing cycles over the operation time of process. Based on the…
This paper considers a mean field game model inspired by crowd motion where agents want to leave a given bounded domain through a part of its boundary in minimal time. Each agent is free to move in any direction, but their maximal speed is bounded in terms of the average density of agents around their position in order to take into account congestion phenomena. After a preliminary study of the corresponding minimal-time optimal control problem, we formulate the mean field game in a Lagrangian setting and prove existence of Lagrangian equilibria using a fixed point strategy. We provide a furthe…
In complex engineered systems, completing an objective is sometimes not enough. The system must be able to reach a set performance characteristic, such as an unmanned aerial vehicle flying from point A to point B, \textit{under 10 seconds}. This introduces the notion of optimality, what is the most efficient, the fastest, the cheapest way to complete a task. This report explores the two pillars of optimal control, Bellman's Dynamic Programming and Pontryagin's Maximum Principle, and compares implementations of both theories onto simulated systems. Dynamic Programming is realized through a Diff…
We prove a version of the stochastic maximum principle, in the sense of Pontryagin, for the finite horizon optimal control of a stochastic partial differential equation driven by an infinite dimensional additive noise. In particular we treat the case in which the non-linear term is of Nemytskii type, dissipative and with polynomial growth. The performance functional to be optimized is fairly general and may depend on point evaluation of the controlled equation. The results can be applied to a large class of non-linear parabolic equations such as reaction-diffusion equations.
We consider the canonical (quantity-based) network revenue management problem, where a firm accepts or rejects incoming customer requests irrevocably in order to maximize expected revenue given limited resources. Due to the curse of dimensionality, the exact solution to this problem by dynamic programming is intractable when the number of resources is large. We study a family of re-solving heuristics that periodically re-optimize an approximation to the original problem known as the deterministic linear program (DLP), where random customer arrivals are replaced by their expectations. We find t…
We use one-step conditional risk mappings to formulate a risk averse version of a total cost problem on a controlled Markov process in discrete time infinite horizon. The nonnegative one step costs are assumed to be lower semi-continuous but not necessarily bounded. We derive the conditions for the existence of the optimal strategies and solve the problem explicitly by giving the robust dynamic programming equations under very mild conditions. We further give an $ε$-optimal approximation to the solution and illustrate our algorithm in two examples of optimal investment and LQ regulator problem…
The classical Method of Successive Approximations (MSA) is an iterative method for solving stochastic control problems and is derived from Pontryagin's optimality principle. It is known that the MSA may fail to converge. Using careful estimates for the backward stochastic differential equation (BSDE) this paper suggests a modification to the MSA algorithm. This modified MSA is shown to converge for general stochastic control problems with control in both the drift and diffusion coefficients. Under some additional assumptions the rate of convergence is shown. The results are valid without restr…
