We consider an optimal switching problem with random lag and possibility of component failure. The random lag is modeled by letting the operation mode follow a regime switching Markov-model with transition intensities that depend on the switching mode. The possibility of failures is modeled by having absorbing components. We show existence of an optimal control for the problem by applying a probabilistic technique based on the concept of Snell envelopes.
We study a zero-sum stochastic differential switching game in infinite horizon. We prove the existence of the value of the game and characterize it as the unique viscosity solution of the associated system of quasi-variational inequalities with bilateral obstacles. We also obtain a verification theorem which provides an optimal strategy of the game. Finally, some numerical examples with two regimes are given.
The leader-following consensus problem for multiple Euler-Lagrange systems was studied recently by the adaptive distributed observer approach under the assumptions that the leader system is neurally stable and the communication network is jointly connected and undirected. In this paper, we will study the same problem without assuming that the leader system is neutrally stable, and the communication network is undirected. The effectiveness of this new result will be illustrated by an example.
This paper is concerned with a stochastic linear-quadratic (LQ) optimal control problem on infinite time horizon, with regime switching, random coefficients, and cone control constraint. To tackle the problem, two new extended stochastic Riccati equations (ESREs) on infinite time horizon are introduced. The existence of the nonnegative solutions, in both standard and singular cases, is proved through a sequence of ESREs on finite time horizon. Based on this result and some approximation techniques, we obtain the optimal state feedback control and optimal value for the stochastic LQ problem exp…
This paper studies a discrete-time optimal switching problem on a finite horizon. The underlying model has a running reward, terminal reward and signed (positive and negative) switching costs. Using the martingale approach to optimal stopping problems, we extend a well known explicit dynamic programming method for computing the value function and the optimal strategy to the case of signed switching costs.
We consider an optimal switching problem where the terminal reward depends on the entire control trajectory. We show existence of an optimal control by applying a probabilistic technique based on the concept of Snell envelopes. We then apply this result to solve an impulse control problem for stochastic delay differential equations driven by a Brownian motion and an independent compound Poisson process. Furthermore, we show that the studied problem arises naturally when maximizing the revenue from operation of a group of hydro-power plants with hydrological coupling.
This paper presents a novel framework which combines a non-iterative solution of Real-Time Nonlinear Receding Horizon Control (NRHC) methodology to achieve consensus within complex network topologies with existing time-delays and in presence of switching topologies. In this formulation, we solve the distributed nonlinear optimization problem for multi-agent network systems directly, \emph{in real-time}, without any dependency on iterative processes, where the stability and convergence guarantees are provided for the solution. Three benchmark examples on non-linear chaotic systems provide valid…
In this paper we study continuous-time two-player zero-sum optimal switching games on a finite horizon. Using the theory of doubly reflected BSDEs with interconnected barriers, we show that this game has a value and an equilibrium in the players' switching controls.
This paper is concerned with optimal switching over multiple modes in continuous time and on a finite horizon. The performance index includes a running reward, terminal reward and switching costs that can belong to a large class of stochastic processes. Particularly, the switching costs are modelled by right-continuous with left-limits processes that are quasi-left-continuous and can take both positive and negative values. We provide sufficient conditions leading to a well known probabilistic representation of the value function for the switching problem in terms of interconnected Snell envelo…
In this paper we show existence and uniqueness of a solution for a system of m variational partial differential inequalities with inter-connected obstacles. This system is the deterministic version of the Verification Theorem of the Markovian optimal m-states switching problem. The switching cost functions are arbitrary. This problem is in relation with the valuation of firms in a financial market.
In this paper, we study a stochastic linear-quadratic control problem with random coefficients and regime switching on a horizon $[0,T\wedgeτ]$, where $τ$ is a given random jump time for the underlying state process and $T$ is a constant. We obtain an explicit optimal state feedback control and explicit optimal cost value by solving a system of stochastic Riccati equations (SREs) with jumps on $[0,T\wedgeτ]$. By the decomposition approach stemming from filtration enlargement theory, we express the solution of the system of SREs with jumps in terms of another system of SREs involving only Brown…
We consider the problem of optimal multiple switching in finite horizon, when the state of the system, including the switching costs, is a general adapted stochastic process. The problem is formulated as an extended impulse control problem and completely solved using probabilistic tools such as the Snell envelop of processes and reflected backward stochastic differential equations. Finally, when the state of the system is a Markov diffusion process, we show that the vector of value functions of the optimal problem is a viscosity solution to a system of variational inequalities with inter-conne…
This paper uses recent results on continuous-time finite-horizon optimal switching problems with negative switching costs to prove the existence of a saddle point in an optimal stopping (Dynkin) game. Sufficient conditions for the game's value to be continuous with respect to the time horizon are obtained using recent results on norm estimates for doubly reflected backward stochastic differential equations. This theory is then demonstrated numerically for the special cases of cancellable call and put options in a Black-Scholes market.
In this paper, we consider a continuous-time mean-variance portfolio selection with regime-switching and random horizon. Unlike previous works, the dynamic of assets are described by non-Markovian regime-switching models in the sense that all the market parameters are predictable with respect to the filtration generated jointly by Markov chain and Brownian motion. We formulate this problem as a constrained stochastic linear-quadratic optimal control problem. The Markov chain is assumed to be independent of the Brownian motion. So the market is incomplete. We derive closed-form expressions for …
This paper studies the problem of the deterministic version of the Verification Theorem for the optimal m-states switching in infinite horizon under Markovian framework with arbitrary switching cost functions. The problem is formulated as an extended impulse control problem and solved by means of probabilistic tools such as the Snell envelop of processes and reflected backward stochastic differential equations. A viscosity solutions approach is employed to carry out a finne analysis on the associated system of m variational inequalities with inter-connected obstacles. We show that the vector o…
In this paper we give a representation formula for the limit of the fnite horizon problem as the horizon becomes infinite, with a nonnegative Lagrangian and unbounded data. It is related to the limit of the discounted infinite horizon problem, as the discount factor goes to zero. We give sufficient conditions to characterize the limit function as unique nonnegative solution of the associated HJB equation. We also briefly discuss the ergodic problem.
This paper studies an online optimization problem with a finite prediction window of cost functions and additional switching costs on decisions. We propose two gradient-based online algorithms: Receding Horizon Gradient Descent (RHGD), and Receding Horizon Accelerated Gradient (RHAG). Both algorithms only require a finite number of projected gradient evaluations at each stage. We provide upper bounds on the dynamic regrets of the proposed algorithms and show that the regret upper bounds decay exponentially with the length of the prediction window. Moreover, we study the fundamental lower bound…
We study so{ù}e infinite-horizon optimization problems on spaces of periodic functions for non periodic Lagrangians. The main strategy relies on the reduction to finite horizon thanks in the introduction of an avering operator.We then provide existence results and necessary optimality conditions in which the corresponding averaged Lagrangian appears.
We study a finite-dimensional continuous-time optimal control problem on finite horizon for a controlled diffusion driven by Brownian motion, in the linear-quadratic case. We admit stochastic coefficients, possibly depending on an underlying independent marked point process, so that our model is general enough to include controlled switching systems where the switching mechanism is not required to be Markovian. The problem is solved by means of a Riccati equation, which a backward stochastic differential equation driven by the Bronwian motion and by the random measure associated to the marked …
We consider the optimization of a dynamical system by switching at discrete time points between abstract evolution equations composed by nonlinearly perturbed strongly continuous semigroups, nonlinear state reset maps at mode transition times and Lagrange-type cost functions including switching costs. In particular, for a fixed sequence of modes, we derive necessary optimality conditions using an adjoint equation based representation for the gradient of the costs with respect to the switching times. For optimization with respect to the mode sequence, we discuss a mode-insertion gradient. The t…
In this paper we study the optimal m-states switching problem in finite horizon as well as infinite horizon with risk of default. We allow the switching cost functionals and cost of default to be of polynomial growth and arbitrary. We show uniqueness of a solution for a system of m variational partial differential inequalities with inter-connected obstacles. This system is the deterministic version of the Verification Theorem of the Markovian optimal m-states switching problem with risk of default. This problem is connected with the valuation of a power plant in the energy market.
This paper studies a system of $m$ variational inequalities with interconnected obstacles in infinite horizon associated to optimal multi-modes switching problems. Our main result is the existence and uniqueness of a continuous solution in viscosity sense, for that system. The proof of the main result strongly relies on the connection between the systems of variational inequalities and reflected backward stochastic differential equations (RBSDEs) with oblique reflection, which will be characterized through a Feynman-Kac's formula. The main feature of our system of infinite horizon RBSDEs is th…
We prove Euler-Lagrange type equations and transversality conditions for generalized infinite horizon problems of the calculus of variations on time scales. Here the Lagrangian depends on the independent variable, an unknown function and its nabla derivative, as well as a nabla indefinite integral that depends on the unknown function.
In this paper we consider an infinite horizon zero-sum differential game where the dynamics of each player and the running cost are also depending on the evolution of some discrete (switching) variables. In particular, such switching variables evolve according to the switching law of a so-called thermostatic delayed relay, applied to the players' states. We first address the problem of the continuity of both lower and upper value function. Then, by a suitable representation of the problem as a coupling of several exit-time differential games, we characterize those value functions as, respectiv…
The paper studies a class of multidimensional optimal stopping problems with infinite horizon for linear switching diffusions. There are two main novelties in the optimal problems considered: the underlying stochastic process has discontinuous paths and the cost function is not necessarily integrable on the entire time horizon, where the latter is often a key assumption in classical optimal stopping theory for diffusions, cf. [22, Corollary 2.9]. Under relatively mild conditions, we show, for the class of multidimensional optimal stopping problems under consideration, that the first entry time…
This paper introduces a new class of optimal switching problems, where the player is allowed to switch at a sequence of exogenous Poisson arrival times, and the underlying switching system is governed by an infinite horizon backward stochastic differential equation system. The value function and the optimal switching strategy are characterized by the solution of the underlying switching system. In a Markovian setting, the paper gives a complete description of the structure of switching regions by means of the comparison principle.
This paper is concerned with the problem of finding the optimal of extraction policies of an oil field in light of various financial and economical restrictions and constraints. Taking into account the fact that the oil price in worldwide commodity markets fluctuates randomly following global and seasonal macro-economic parameters, we model the evolution of the oil price as a mean reverting regime switching jump diffusion process. We formulate this problem as finite-time horizon optimal control problem. We solve the control problem using the method of viscosity solutions. Moreover, we construc…
Switching time optimization arises in finite-horizon optimal control for switched systems where, given a sequence of continuous dynamics, one minimizes a cost function with respect to the switching times. We propose an efficient method for computing the optimal switching times for switched linear and nonlinear systems. A novel second-order optimization algorithm is introduced where, at each iteration, the dynamics are linearized over an underlying time grid to compute the cost function, the gradient and the Hessian efficiently. With the proposed method, the most expensive operations at each it…
This paper deals with an isoperimetric optimal control problem for nonlinear control-affine systems with periodic boundary conditions. As it was shown previously, the candidates for optimal controls for this problem can be obtained within the class of bang-bang input functions. We consider a parametrization of these inputs in terms of switching times. The control-affine system under consideration is transformed into a driftless system by assuming that the controls possess properties of a partition of unity. Then the problem of constructing periodic trajectories is studied analytically by apply…
This paper studies an optimal stochastic impulse control problem in a finite horizon with a decision lag, by which we mean that after an impulse is made, a fixed number units of time has to be elapsed before the next impulse is allowed to be made. The continuity of the value function is proved. A suitable version of dynamic programming principle is established, which takes into account the dependence of state process on the elapsed time. The corresponding Hamilton-Jacobi-Bellman (HJB) equation is derived, which exhibit some special feature of the problem. The value function of this optimal imp…
We consider problems of the calculus of variations on unbounded time scales. We prove the validity of the Euler-Lagrange equation on time scales for infinite horizon problems, and a new transversality condition.
We address the problem of making a managerial decision when the investment project is subsidized, which results in the resolution of an infinite-horizon optimal stopping problem of a switching diffusion driven by either an homogeneous or an inhomogeneous continuous-time Markov chain. We provide a characterization of the value function (and optimal strategy) of the optimal stopping problem. On the one hand, broadly, we can prove that the value function is the unique viscosity solution to a system of HJB equations. On the other hand, when the Markov chain is homogeneous and the switching diffusi…
We study the regularity of the stochastic representation of the solution of a class of initial-boundary value problems related to a regime-switching diffusion. This representation is related to the value function of a finite-horizon optimal stopping problem such as the price of an American-style option in finance. We show continuity and smoothness of the value function using coupling and time-change techniques. As an application, we find the minimal payoff scenario for the holder of an American-style option in the presence of regime-switching uncertainty under the assumption that the transitio…
We consider the classical stochastic multi-armed bandit problem with a constraint that limits the total cost incurred by switching between actions to be no larger than a given switching budget. For this problem, we prove matching upper and lower bounds on the optimal (i.e., minimax) regret, and provide efficient rate-optimal algorithms. Surprisingly, the optimal regret of this problem exhibits a non-conventional growth rate in terms of the time horizon and the number of arms. Consequently, we discover surprising "phase transitions" regarding how the optimal regret rate changes with respect to …
We solve non-Markovian optimal switching problems in discrete time on an infinite horizon, when the decision maker is risk aware and the filtration is general, and establish existence and uniqueness of solutions for the associated reflected backward stochastic difference equations. An example application to hydropower planning is provided.
We consider an optimal control problem for the three-dimensional non-linear Primitive Equations of the ocean in a vertically bounded and horizontally periodic domain. The observation operator maps a solution of the Primitive Equations to the trajectory of a Lagrangian particle. This paper proves the existence of an optimal control for the regularized problem. To do that, we prove also new energy estimates for the Primitive Equations, thanks to well-chosen functional spaces, which distinguish the vertical dimension from the horizontal ones. We illustrate the result with a numerical experiment.
