In this paper, we study an optimal exit time problem with general running and terminal costs and a target $\mathcal{S}\subset\mathbb{R}^d$ having an inner ball property for a nonlinear control system that satisfies mild controllability assumptions. In particular, Petrov's condition at the boundary of $\mathcal{S}$ is not required and the value function $V$ may fail to be locally Lipschitz. In such a weakened set-up, we first establish a representation formula for proximal (horizontal) supergradients of $V$ by using transported proximal normal vectors. This allows us to obtain an external sphere condition for the hypograph of $V$ which yields several regularity properties. In particular, $V$ is almost everywhere twice differentiable and the Hausdorff dimension of its singularities is not greater than $d-1/2$. Furthermore, besides optimality conditions for trajectories of the optimal control problem, we extend the analysis to propagation of singularities and differentiability properties of the value function. An upper bound for the Hausdorff measure of the singular set is also studied, which implies that $V$ is a function of special bounded variation.
We consider a class of exit--time control problems for nonlinear systems with a nonnegative vanishing Lagrangian. In general, the associated PDE may have multiple solutions, and known regularity and stability properties do not hold. In this paper we obtain such properties and a uniqueness result under some explicit sufficient conditions. We briefly investigate also the infinite horizon problem.
We employ a natural method from the perspective of the optimal stopping theory to analyze entry-exit decisions with implementation delay of a project, and provide closed expressions for optimal entry decision times, optimal exit decision times and the maximal expected present value from the project. The results in conventional research were obtained under the restriction that the sum of the entry cost and the exit cost is nonnegative. In practice, we usually meet this sum is negative, so it is necessary to remove the restriction. If the sum is negative, there may exist two price triggers of en…
We consider the control problem with \textit{exit time}. Unlike the Bolza and Mayer problems, in this problem the terminal time of the trajectories is not fixed, but it is the first time at which they reach a given closed subset - \textit{the target}. The most studied example is the \textit{optimal time problem}, where we want to steer a point to the target in minimal time. In this section, we first introduce the exit time problem, then we recall the existence of optimal controls, and some regularity results for the value function. We then use a suitable form of the \textit{Pontryagin maximum …
The main result in this paper is a variational formula for the exit rate from a bounded domain for a diffusion process in terms of the stationary law of the diffusion constrained to remain in this domain forever. Related results on the geometric ergodicity of the controlled Q-process are also presented.
In this paper we propose a new way of proving the value of a firm that is currently producing a certain product and faces the option to exit the market. The problem of optimal exiting is an optimal stopping problem, that can be solved using the dynamic programming principle. This approach leads to a partial differential equation, called the Hamilton-Jacobi-Bellman equation. This is a free-boundary problem, and therefore, we propose an approximation for the original model. We prove the convergence of the solution of the approximated problem to the original one and finally, using the Implicit Fu…
We study the Hamilton-Jacobi equation for undiscounted exit time control problems with general nonnegative Lagrangians using the dynamic programming approach. We prove theorems characterizing the value function as the unique bounded-from-below viscosity solution of the Hamilton-Jacobi equation which is null on the target. The result applies to problems with the property that all trajectories satisfying a certain integral condition must stay in a bounded set. We allow problems for which the Lagrangian is not uniformly bounded below by positive constants, in which the hypotheses of the known uni…
We develop a model for the industry dynamics in the electricity market, based on mean-field games of optimal stopping. In our model, there are two types of agents: the renewable producers and the conventional producers. The renewable producers choose the optimal moment to build new renewable plants, and the conventional producers choose the optimal moment to exit the market. The agents interact through the market price, determined by matching the aggregate supply of the two types of producers with an exogenous demand function. Using a relaxed formulation of optimal stopping mean-field games, w…
We consider an optimal control on networks in the spirit of the works of Achdou et al. (2013) and Imbert et al. (2013). The main new feature is that there are entry (or exit) costs at the edges of the network leading to a possible discontinuous value function. We characterize the value function as the unique viscosity solution of a new Hamilton-Jacobi system. The uniqueness is a consequence of a comparison principle for which we give two different proofs, one with arguments from the theory of optimal control inspired by Achdou et al. (2014) and one based on partial differential equations techn…
This paper studies a mean field game inspired by crowd motion in which agents evolve in a compact domain and want to reach its boundary minimizing the sum of their travel time and a given boundary cost. Interactions between agents occur through their dynamic, which depends on the distribution of all agents. We start by considering the associated optimal control problem, showing that semi-concavity in space of the corresponding value function can be obtained by requiring as time regularity only a lower Lipschitz bound on the dynamics. We also prove differentiability of the value function along …
Even in the face of deteriorating and highly volatile demand, firms often invest in, rather than discard, aging technologies. In order to study this phenomenon, we model the firm's profit stream as a Brownian motion with negative drift. At each point in time, the firm can continue operations, or it can stop and exit the project. In addition, there is a one-time option to make an investment which boosts the project's profit rate. Using stochastic analysis, we show that the optimal policy always exists and that it is characterized by three thresholds. There are investment and exit thresholds bef…
We study the phenomenon of loss of lock in the optimal non-causal phase estimation problem, a benchmark problem in nonlinear estimation. Our method is based on the computation of the asymptotic distribution of the optimal estimation error in case the number of trajectories in the optimization problem is finite. The computation is based directly on the minimum noise energy optimality criterion rather than on state equations of the error, as is the usual case in the literature. The results include an asymptotic computation of the mean time to lose lock (MTLL) in the optimal smoother. We show tha…
We consider a stochastic optimal exit time feedback control problem. The Bellman equation is solved approximatively via the Policy Iteration algorithm on a polynomial ansatz space by a sequence of linear equations. As high degree multi-polynomials are needed, the corresponding equations suffer from the curse of dimensionality even in moderate dimensions. We employ tensor-train methods to account for this problem. The approximation process within the Policy Iteration is done via a Least-Squares ansatz and the integration is done via Monte-Carlo methods. Numerical evidences are given for the (mu…
We develop policy gradients methods for stochastic control with exit time in a model-free setting. We propose two types of algorithms for learning either directly the optimal policy or by learning alternately the value function (critic) and the optimal control (actor). The use of randomized policies is crucial for overcoming notably the issue related to the exit time in the gradient computation. We demonstrate the effectiveness of our approach by implementing our numerical schemes in the application to the problem of share repurchase pricing. Our results show that the proposed policy gradient …
We consider a kind of stochastic exit time optimal control problems, in which the cost function is defined through a nonlinear backward stochastic differential equation. We study the regularity of the value function for such a control problem. Then extending Peng's backward semigroup method, we show the dynamic programming principle. Moreover, we prove that the value function is a viscosity solution to the following generalized Hamilton-Jacobi-Bellman equation with Dirichlet boundary: \[ \left\{ \begin{array} [c]{l} \inf\limits_{v\in V}\left\{\mathcal{L}(x,v)u(x)+f(x,u(x),\nabla u(x) σ(x,v),v)…
We consider a class of exit time stochastic control problems for diffusion processes with discounted criterion, where the controller can utilize a given amount of resource, called "fuel". In contrast to the vast majority of existing literature, concerning the "finite fuel" problems, it is assumed that the intensity of fuel consumption is bounded. We characterize the value function of the optimization problem as the unique continuous viscosity solution of the Dirichlet boundary value problem for the correspondent Hamilton-Jacobi-Bellman (HJB) equation. Our assumptions concern the HJB equations,…
We apply the Stochastic Perron method, created by Bayraktar and Sîrbu, to a stochastic exit time control problem. Our main assumption is the validity of the Strong Comparison Result for the related Hamilton-Jacobi-Bellman (HJB) equation. Without relying on Bellman's optimality principle we prove that inside the domain the value function is continuous and coincides with a viscosity solution of the Dirichlet boundary value problem for the HJB equation.
In this paper we present an optimal control approach modeling fast exit scenarios in pedestrian crowds. In particular we consider the case of a large human crowd trying to exit a room as fast as possible. The motion of every pedestrian is determined by minimizing a cost functional, which depends on his/her position, velocity, exit time and the overall density of people. This microscopic setup leads in the mean-field limit to a parabolic optimal control problem. We discuss the modeling of the macroscopic optimal control approach and show how the optimal conditions relate to Hughes model for ped…
We determine a weaker sufficient condition than that of Theorem 5.2.1 in Fleming and Soner (2006) for the continuity of the value functions of stochastic exit time control problems.
In this paper, we consider an optimal control problem for a prescription opioid epidemic model that describes the interaction between the regular prescription or addictive use of opioid drugs, and the process of rehabilitation and that of relapsing into opioid drug use. In particular, our interest is in the situation, where the control appearing linearly in the opioid epidemics is interpreted as the rate at which the susceptible individuals are effectively removed from the population due to an opioid-related intervention policy or when the dynamics of the addicted is strategically influenced d…
We study sequences, parametrized by the number of agents, of many agent exit time stochastic control problems with risk-sensitive cost structure. We identify a fully characterizing assumption, under which each of such control problem corresponds to a risk-neutral stochastic control problem with additive cost, and sequentially to a risk-neutral stochastic control problem on the simplex, where the specific information about the state of each agent can be discarded. We also prove that, under some additional assumptions, the sequence of value functions converges to the value function of a determin…
We study the impact of learning on the optimal policy and the time-to-decision in an infinite-horizon Bayesian sequential decision model with two irreversible alternatives, exit and expansion. In our model, a firm undertakes a small-scale pilot project so as to learn, via Bayesian updating, about the project\textquoteright s profitability, which is known to be in one of two possible states. The firm continuously observes the project\textquoteright s cumulative profit, but the true state of the profitability is not immediately revealed because of the inherent noise in the profit stream. The fir…
