We consider a class of dynamic advertising problems under uncertainty in the presence of carryover and distributed forgetting effects, generalizing a classical model of Nerlove and Arrow. In particular, we allow the dynamics of the product goodwill to depend on its past values, as well as previous advertising levels. Building on previous work of two of the authors, the optimal advertising model is formulated as an infinite dimensional stochastic control problem. We obtain (partial) regularity as well as approximation results for the corresponding value function. Under specific structural assumptions we study the effects of delays on the value function and optimal strategy. In the absence of carryover effects, since the value function and the optimal advertising policy can be characterized in terms of the solution of the associated HJB equation, we obtain sharper characterizations of the optimal policy.
We consider a class of optimal control problems of stochastic delay differential equations (SDDE) that arise in connection with optimal advertising under uncertainty for the introduction of a new product to the market, generalizing classical work of Nerlove and Arrow (1962). In particular, we deal with controlled SDDE where the delay enters both the state and the control. Following ideas of Vinter and Kwong (1981) (which however hold only in the deterministic case), we reformulate the problem as an infinite dimensional stochastic control problem to which we associate, through the dynamic progr…
Stochastic optimal control problems with delays in the state and in the control via viscosity solutions and applications to optimal advertising and optimal investment problems
We propose a novel approach to modeling advertising dynamics for a firm operating over distributed market domain based on controlled partial differential equations of diffusion type. Using our model, we consider a general type of finite-horizon profit maximization problem in a monopoly setting. By reformulating this profit maximization problem as an optimal control problem in infinite dimensions, we derive sufficient conditions for the existence of its optimal solutions under general profit functions, as well as state and control constraints, and provide general characterization of the optimal…
The response to a setpoint change in PID controlled feedback systems plays an important role for the tuning methods. This response may be easily evaluated in linear systems without delay by solving the related ordinary differential equations. These equations are also obtained for system with delay, if the delay exponential term is approximated to a polynomial by one of the Pade expressions. If this simplification is avoided, one must deal with differential difference equations. In this paper their explicit solution, obtained with the method of steps, is presented. Moreover a second order equat…
The stability of feedback systems consisting of linear time-delay plants and PID controllers has been investigated for many years by means of several methods, of which the Nyquist criterion, a generalization of the Hermite-Biehler Theorem, and the root location method are well known. The main purpose of these researches is to determine the range of controller parameters that allow stability. Explicit and complete expressions of the boundaries of these regions and computation procedures with a finite number of steps are now available only for first-order plants, provided with one time delay. In…
In this manuscript we consider a class optimal control problem for stochastic differential delay equations. First, we rewrite the problem in a suitable infinite-dimensional Hilbert space. Then, using the dynamic programming approach, we characterize the value function of the problem as the unique viscosity solution of the associated infinite-dimensional Hamilton-Jacobi-Bellman equation. Finally, we prove a $C^{1,α}$-partial regularity of the value function. We apply these results to path dependent financial and economic problems (Merton-like portfolio problem and optimal advertising).
In this paper we consider a family of optimal control problems for economic models whose state variables are driven by Delay Differential Equations (DDE's). We consider two main examples: an AK model with vintage capital and an advertising model with delay effect. These problems are very difficult to treat for three main reasons: the presence of the DDE's, that makes them infinite dimensional; the presence of state constraints; the presence of delay in the control. Our main goal is to develop, at a first stage, the Dynamic Programming approach for this family of problems. The Dynamic Programmi…
We consider a controlled quantum system whose finite dimensional state is governed by a discrete-time nonlinear Markov process. In open-loop, the measurements are assumed to be quantum non-demolition (QND). The eigenstates of the measured observable are thus the open-loop stationary states: they are used to construct a closed-loop supermartingale playing the role of a strict control Lyapunov function. The parameters of this supermartingale are calculated by inverting a Metzler matrix that characterizes the impact of the control input on the Kraus operators defining the Markov process. The resu…
As a main step in the numerical solution of control problems in continuous time, the controlled process is approximated by sequences of controlled Markov chains, thus discretising time and space. A new feature in this context is to allow for delay in the dynamics. The existence of an optimal strategy with respect to the cost functional can be guaranteed in the class of relaxed controls. Weak convergence of the approximating extended Markov chains to the original process together with convergence of the associated optimal strategies is established.
We consider a one-dimensional controlled reaction-diffusion equation, where the control acts on the boundary and is subject to a constant delay. Such a model is a paradigm for more general parabolic systems coupled with a transport equation. We prove that this is possible to stabilize (in H 1 norm) this process by means of an explicit predictor-based feedback control that is designed from a finite-dimensional subsystem. The implementation is very simple and efficient and is based on standard tools of pole-shifting. Our feedback acts on the system as a finite-dimensional predictor. We compare o…
We consider optimal control of decentralized LQG problems for plants controlled by two players having asymmetric information sharing patterns between them. In one scenario, players are assumed to have a bidirectional error-free, unlimited rate communication channel with no delay in one direction and a unit delay in the other. In another scenario, the communication channel is assumed to be unidirectional with a unit delay. Delayed information sharing patterns in general do not admit linear optimal control laws and are thus difficult to control optimally. However, in these scenarios, we show tha…
Distributed averaging-based integral (DAI) controllers are becoming increasingly popular in power system applications. The literature has thus far primarily focused on disturbance rejection, steady-state optimality and adaption to complex physical system models without considering uncertainties on the cyber and communication layer nor their effect on robustness and performance. In this paper, we derive sufficient delay-dependent conditions for robust stability of a secondary-frequency-DAI-controlled power system with respect to heterogeneous communication delays, link failures and packet losse…
We consider an optimal switching problem where the terminal reward depends on the entire control trajectory. We show existence of an optimal control by applying a probabilistic technique based on the concept of Snell envelopes. We then apply this result to solve an impulse control problem for stochastic delay differential equations driven by a Brownian motion and an independent compound Poisson process. Furthermore, we show that the studied problem arises naturally when maximizing the revenue from operation of a group of hydro-power plants with hydrological coupling.
For the sake of saving time and costs the feedback control based on discrete-time observations is used to stabilize the switching diffusion systems. Response lags are required by most of physical systems and play a key role in the feedback control. The aim of this paper is to design delay feedback control functions based on the discrete-time observations of the system states and the Markovian states in order for the controlled switching diffusion system (SDS) to be exponentially stable in $p$th moment and probability one as well as stable in $H_\infty$. The designed control principles are impl…
Multi-objective LQR with Optimum Weight Selection to Design FOPID Controllers for Delayed Fractional Order Processes
School of Electronics and Computer Science, University of Southampton（南安普顿大学电子与计算机科学学院）；Department of Power Engineering, Jadavpur University（贾达普大学动力工程系）；Department of Earth Science and Engineering, Imperial College London（帝国理工学院地球科学与工程系）；Reactor Control Division, Bhabha Atomic Research Centre（巴哈原子研究中心反应堆控制部）
