Stochastic control problems related to optimal advertising under uncertainty are considered. In particular, we determine the optimal strategies for the problem of maximizing the utility of goodwill at launch time and minimizing the disutility of a stream of advertising costs that extends until the launch time for some classes of stochastic perturbations of the classical Nerlove-Arrow dynamics. We also consider some generalizations such as problems with constrained budget and with discretionary launching.
This paper deals with a class of optimal control problems which arises in advertising models with Volterra Ornstein-Uhlenbeck process representing the product goodwill. Such choice of the model can be regarded as a stochastic modification of the classical Nerlove-Arrow model that allows to incorporate both presence of uncertainty and empirically observed memory effects such as carryover or distributed forgetting. We present an approach to solve such optimal control problems based on an infinite dimensional lift which allows us to recover Markov properties by formulating a optimization problem …
The main purpose of this paper is devoted to an introduction of the stochastic tensor complementarity problem. We consider the expected residual minimization formulation of the stochastic tensor complementarity problem. We show that the solution set of the expected residual minimization problem is nonempty and bounded, if the associated tensor is an $R_0$ tensor. We also prove that the associated tensor being a stochastic $R_0$ tensor is a necessary and sufficient condition for the solution set of the expected residual minimization problem to be nonempty and bounded.
We study the closed-loop solvability of a stochastic linear quadratic optimal control problem for systems governed by stochastic evolution equations. This solvability is established by means of solvability of the corresponding Riccati equation, which is implied by the uniform convexity of the quadratic cost functional. At last, conditions ensuring the uniform convexity of the cost functional are discussed.
In this paper, we study the maximum principle for stochastic optimal control problems of forward-backward stochastic difference systems (FBSΔSs). Two types of FBSΔSs are investigated. The first one is described by a partially coupled forward-backward stochastic difference equation (FBSΔE) and the second one is described by a fully coupled FBSΔE. By adopting an appropriate representation of the product rule and an appropriate formulation of the backward stochastic difference equation (BSΔE), we deduce the adjoint difference equation. Finally, the maximum principle for this optimal control probl…
We establish, for the first time, a connection between stochastic games and multiparameter eigenvalue problems, using the theory developed by Shapley and Snow (1950). This connection provides new results, new proofs, and new tools for studying stochastic games.
We consider the control problem of the stochastic Navier-Stokes equations in multidimensional domains introduced in \cite{ocpc} restricted to noise terms defined by Q-Wiener processes. Using a stochastic maximum principle, we derive a necessary optimality condition to design the optimal control based on an adjoint equation, which is given by a backward SPDE. Moreover, we show that the optimal control satisfies a sufficient optimality condition. As a consequence, we can solve uniquely control problems constrained by the stochastic Navier-Stokes equations especially for two-dimensional as well a…
In this paper, we study the stochastic linear complementarity problems on extended second order cones (stochastic ESOCLCP). We first convert the problem to a stochastic mixed complementarity problem on the nonegative orthant (SMixCP). Enlightened by the idea of Chen and Lin(2011), we introduce the Conditional Value-at-risk (CVaR) method to measure the loss of complementarity in the stochastic case. A CVaR - based minimisation problem is introduced to achieve a solution which is "good enough" for the complementarity requirement of the original SMixCP. Smoothing function and sample average appro…
In this study, we consider an optimal control problem driven by a stochastic differential system with a stopping time terminal cost functional. We establish the stochastic maximum principle for this new kind of an optimal control problem by introducing a discrete terminal system. Finally, we provide an example to describe the main results of this study.
Long-Time Behaviors of Stochastic Linear-Quadratic Optimal Control Problems
University of Michigan（密歇根大学）；California State University San Marcos（圣马科斯加州州立大学）；Worcester Polytechnic Institute（伍斯特理工学院）；University of Central Florida（中佛罗里达大学）
It is a longstanding unsolved problem to characterize the optimal feedback controls for general linear quadratic optimal control problem of stochastic evolution equation with random coefficients. A solution to this problem is given in [21] under some assumptions which can be verified for interesting concrete models, such as controlled stochastic wave equations, controlled stochastic Schrödinger equations, etc. More precisely, the authors establish the equivalence between the existence of optimal feedback operator and the solvability of the corresponding operator-valued, backward stochastic Ric…
In this paper we consider discrete time stochastic optimal control problems over infinite and finite time horizons. We show that for a large class of such problems the Taylor polynomials of the solutions to the associated Dynamic Programming Equations can be computed degree by degree.
This paper studies a continuous-time stochastic linear-quadratic (SLQ) optimal control problem on infinite-horizon. A data-driven policy iteration algorithm is proposed to solve the SLQ problem. Without knowing three system coefficient matrices, this algorithm uses the collected data to iteratively approximate a solution of the corresponding stochastic algebraic Riccati equation (SARE). A simulation example is provided to illustrate the effectiveness and applicability of the algorithm.
This paper addresses risk awareness of stochastic optimization problems. Nested risk measures appear naturally in this context, as they allow beneficial reformulations for algorithmic treatments. The reformulations presented extend usual Hamilton-Jacobi-Bellman equations in dynamic optimization by involving risk awareness in the problem formulation. Nested risk measures are built on risk measures, which originate by conditioning on the history of a stochastic process. We derive martingale properties of these risk measures and use them to prove continuity. It is demonstrated that stochastic opt…
In this paper, we establish two Carleman estimates for a stochastic degenerate parabolic equation. The first one is for the backward stochastic degenerate parabolic equation with singular weight function. Combining this Carleman estimate and an approximate argument, we prove the null controllability of the forward stochastic degenerate parabolic equation with the gradient term. The second one is for the forward stochastic degenerate parabolic equation with regular weighted function, based on which we obtain the Lipschitz stability for an inverse problem of determining a random source depending…
In this paper, we study the optimal control of a discrete-time stochastic differential equation (SDE) of mean-field type, where the coefficients can depend on both a function of the law and the state of the process. We establish a new version of the maximum principle for discrete-time stochastic optimal control problems. Moreover, the cost functional is also of the mean-field type. This maximum principle differs from the classical principle since we introduce new discrete-time backward (matrix) stochastic equations. Based on the discrete-time backward stochastic equations where the adjoint equ…
This paper aims to explore the relationship between maximum principle and dynamic programming principle for stochastic recursive control problem with random coefficients. Under certain regular conditions for the coefficients, the relationship between the Hamilton system with random coefficients and stochastic Hamilton-Jacobi-Bellman equation is obtained. It is very different from the deterministic coefficients case since stochastic Hamilton-Jacobi-Bellman equation is a backward stochastic partial differential equation with solution being a pair of random fields rather than a deterministic func…
In this paper, we consider linear quadratic optimal control with mean-field type for discrete-time stochastic systems with state and control dependent noise. An optimal control problem is studied for a linear mean-field stochastic differential equation with a quadratic cost functional. The coefficients and the weighting matrices in the cost functional are all assumed to be deterministic.
In this paper, we first introduce a new spatial-temporal interaction operator to describe the space-time dependent phenomena. Then we consider the stochastic optimal control of a new system governed by a stochastic partial differential equation with the spatial-temporal interaction operator. To solve such a stochastic optimal control problem, we derive an adjoint backward stochastic partial differential equation with spatial-temporal dependence by defining a Hamiltonian functional, and give both the sufficient and necessary (Pontryagin-Bismut-Bensoussan type) maximum principles. Moreover, the …
In this paper, we consider a stochastic decision problem for a system governed by a stochastic differential equation, in which an optimal decision is made in such a way to minimize a vector-valued accumulated cost over a finite-time horizon that is associated with the solution of a certain multi-dimensional backward stochastic differential equation (BSDE). Here, we also assume that the solution for such a multi-dimensional BSDE {\it almost surely} satisfies a backward stochastic viability property w.r.t. a given closed convex set. Moreover, under suitable conditions, we establish the existence…
We consider the stochastic variational inequality problem in which the map is expectation-valued in a component-wise sense. Much of the available convergence theory and rate statements for stochastic approximation schemes are limited to monotone maps. However, non-monotone stochastic variational inequality problems are not uncommon and are seen to arise from product pricing, fractional optimization problems, and subclasses of economic equilibrium problems. Motivated by the need to address a broader class of maps, we make the following contributions: (i) We present an extragradient-based stocha…
This paper is concerned with the stochastic linear-quadratic optimal control problem with Poisson jumps. The coefficients in the state equation and the weighting matrices in the cost functional are all deterministic but are allowed indefinite. The notion of closed-loop strategies is introduced, and the optimal closed-loop strategy is characterized by a Riccati integral-differential equation and a backward stochastic differential equation with Poisson jumps.
In this paper, we consider a null controllability and an inverse source problem for stochastic Grushin equation with boundary degeneracy and singularity. We construct two special weight functions to establish two Carleman estimates for the whole stochastic Grushin operator with singular potential by a weighted identity method. One is for the backward stochastic Grushin equation with singular weight function. We then apply it to prove the null controllability for stochastic Grushin equation for any $T$ and any degeneracy $γ>0$, when our control domain touches the degeneracy line $\{x=0\}$. In o…
We study constrained nested stochastic optimization problems in which the objective function is a composition of two smooth functions whose exact values and derivatives are not available. We propose a single time-scale stochastic approximation algorithm, which we call the Nested Averaged Stochastic Approximation (NASA), to find an approximate stationary point of the problem. The algorithm has two auxiliary averaged sequences (filters) which estimate the gradient of the composite objective function and the inner function value. By using a special Lyapunov function, we show that NASA achieves th…
Multiobjective stochastic programming is a field well located to tackle problems arising in emergencies, given that uncertainty and multiple objectives are usually present in such problems. A new concept of solution is proposed in this work, especially designed for risk-aversion solutions. A linear programming model is presented to obtain such solution.
In this paper, we obtain the maximum principle for optimal controls of stochastic systems with jumps by introducing a new method of variation. The control is allowed to enter both diffusion and jump term and the control domain need not to be convex.
