We study the stochastic control-stopping problem when the data are of polynomial growth. The approach is based on backward stochastic dierential equations (BSDEs for short). The problem turns into the study of a specic reected BSDE with a stochastic Lipschitz coecient for which we show existence and uniqueness of the solution. We then establish its relationship with the value function of the control-stopping problem. The optimal strategy is exhibited. Finally in the Markovian framework we prove that the value function is the unique viscosity solution of the associated Hamilton-Jacobi-Bellman equation.
In this paper we study a representation problem first considered in a simpler version by Bank and El Karoui [2004]. A key ingredient to this problem is a random measure $μ$ on the time axis which in the present paper is allowed to have atoms. Such atoms turn out to not only pose serious technical challenges in the proof of the representation theorem, but actually have significant meaning in its applications, for instance, in irreversible investment problems. These applications also suggest to study the problem for processes which are measurable with respect to a Meyer-$σ$-field that lies betwe…
We consider a zero-sum stochastic game for continuous-time Markov chain with countable state space and unbounded transition and pay-off rates. The additional feature of the game is that the controllers together with taking actions are also allowed to stop the process. Under suitable hypothesis we show that the game has a value and it is the unique solution of certain dynamic programming inequalities with bilateral constraints. In the process we also prescribe a saddle point equilibrium.
Given a stochastic state process $(X_t)_t$ and a real-valued submartingale cost process $(S_t)_t$, we characterize optimal stopping times $τ$ that minimize the expectation of $S_τ$ while realizing given initial and target distributions $μ$ and $ν$, i.e., $X_0\sim μ$ and $X_τ\sim ν$. A dual optimization problem is considered and shown to be attained under suitable conditions. The optimal solution of the dual problem then provides a contact set, which characterizes the location where optimal stopping can occur. The optimal stopping time is uniquely determined as the first hitting time of this co…
In this study, we consider an optimal control problem driven by a stochastic differential system with a stopping time terminal cost functional. We establish the stochastic maximum principle for this new kind of an optimal control problem by introducing a discrete terminal system. Finally, we provide an example to describe the main results of this study.
We consider a non-Markovian optimal stopping problem on finite horizon. We prove that the value process can be represented by means of a backward stochastic differential equation (BSDE), defined on an enlarged probability space, containing a stochastic integral having a one-jump point process as integrator and an (unknown) process with a sign constraint as integrand. This provides an alternative representation with respect to the classical one given by a reflected BSDE. The connection between the two BSDEs is also clarified. Finally, we prove that the value of the optimal stopping problem is t…
Bellman formulated a vague principle for optimization over time, which characterizes optimal policies by stating that a decision maker should not regret previous decisions retrospectively. This paper addresses time consistency in stochastic optimization. The problem is stated in generality first. The paper discusses time consistent decision-making by addressing risk measures which are recursive, nested, dynamically or time consistent and introduces stopping time risk measures. It turns out that the paradigm of time consistency is in conflict with various desirable, classical properties of gene…
In this paper we establish a new connection between a class of 2-player nonzero-sum games of optimal stopping and certain $2$-player nonzero-sum games of singular control. We show that whenever a Nash equilibrium in the game of stopping is attained by hitting times at two separate boundaries, then such boundaries also trigger a Nash equilibrium in the game of singular control. Moreover a differential link between the players' value functions holds across the two games.
This paper analyses a stochastic differential game of control and stopping in which one of the players modifies a diffusion process using impulse controls, an adversary then chooses a stopping time to end the game. The paper firstly establishes the regularity and boundedness of the upper and lower value functions from which an appropriate variant of the dynamic programming principle (DPP) is derived. It is then proven that the upper and lower value functions coincide so that the game admits a value and that the value of the game is a unique viscosity solution to a HJBI equation described by a …
We prove the dynamic programming principle (DPP) in a class of problems where an agent controls a $d$-dimensional diffusive dynamics via both classical and singular controls and, moreover, is able to terminate the optimisation at a time of her choosing, prior to a given maturity. The time-horizon of the problem is random and it is the smallest between a fixed terminal time and the first exit time of the state dynamics from a Borel set. We consider both the cases in which the total available fuel for the singular control is either bounded or unbounded. We build upon existing proofs of DPP and e…
The purpose of this paper is to study the following topics and the relation between them: (i) Optimal singular control of mean-field stochastic differential equations with memory, (ii) reflected advanced mean-field backward stochastic differential equations, and (iii) optimal stopping of mean-field stochastic differential equations. More specifically, we do the following: - We prove the existence and uniqueness of the solutions of some reflected advanced memory backward stochastic differential equations (AMBSDEs), - we give sufficient and necessary conditions for an optimal singular control of…
We study the problem of optimal stopping of conditional McKean-Vlasov (mean-field) stochastic differential equations with jumps (conditional McKean-Vlasov jump diffusions, for short). We obtain sufficient variational inequalities for a function to be the value function of such a problem and for a stopping time to be optimal. To achieve this, we combine the state equation for the conditional McKean-Vlasov equation with the associated stochastic Fokker-Planck equation for the conditional law of the solution of the state. This gives us a Markovian system which can be handled by using a version of…
We present a novel method for solving a class of time-inconsistent optimal stopping problems by reducing them to a family of standard stochastic optimal control problems. In particular, we convert an optimal stopping problem with a non-linear function of the expected stopping time in the objective into optimization over an auxiliary value function for a standard stochastic control problem with an additional state variable. This approach differs from the previous literature which primarily employs Lagrange multiplier methods or relies on exact solutions. In contrast, we characterize the auxilia…
The purpose of this paper is two-fold: We extend the well-known relation between optimal stopping and randomized stopping of a given stochastic process to a situation where the available information flow is a filtration with no a priori assumed relation to the filtration of the process. We call these problems optimal stopping and randomized stopping with general information. Following an idea of Krylov [K] we introduce a special singular stochastic control problem with general information and show that this is also equivalent to the partial information optimal stopping and randomized stopping …
We provide a new probabilistic proof of the connection between Rost's solution of the Skorokhod embedding problem and a suitable family of optimal stopping problems for Brownian motion with finite time-horizon. In particular we use stochastic calculus to show that the time reversal of the optimal stopping sets for such problems forms the so-called Rost's reversed barrier.
Stopping criteria for Stochastic Gradient Descent (SGD) methods play important roles from enabling adaptive step size schemes to providing rigor for downstream analyses such as asymptotic inference. Unfortunately, current stopping criteria for SGD methods are often heuristics that rely on asymptotic normality results or convergence to stationary distributions, which may fail to exist for nonconvex functions and, thereby, limit the applicability of such stopping criteria. To address this issue, in this work, we rigorously develop two stopping criteria for SGD that can be applied to a broad clas…
We consider a class of time-inhomogeneous optimal stopping problems and we provide sufficient conditions on the data of the problem that guarantee monotonicity of the optimal stopping boundary. In our setting, time-inhomogeneity stems not only from the reward function but, in particular, from the time dependence of the drift coefficient of the one-dimensional stochastic differential equation (SDE) which drives the stopping problem. In order to obtain our results, we mostly employ probabilistic arguments: we use a comparison principle between solutions of the SDE computed at different starting …
Nonlinear Stochastic Optimal Control and Optimal Stopping using the Fokker-Planck Transformation
Daniel Guggenheim School of Aerospace, Georgia Institute of Technology（佐治亚理工学院丹尼尔·古根海姆航空学院）；Department of Mechanical Engineering, University of Alabama（阿拉巴马大学机械工程系）
Optimal stopping is a fundamental class of stochastic dynamic optimization problems with numerous applications in finance and operations management. We introduce a new approach for solving computationally-demanding stochastic optimal stopping problems with known probability distributions. The approach uses simulation to construct a robust optimization problem that approximates the stochastic optimal stopping problem to any arbitrary accuracy; we then solve the robust optimization problem to obtain near-optimal Markovian stopping rules for the stochastic optimal stopping problem. In this paper,…
Equivalences are known between problems of singular stochastic control (SSC) with convex performance criteria and related questions of optimal stopping, see for example Karatzas and Shreve [SIAM J. Control Optim. 22 (1984)]. The aim of this paper is to investigate how far connections of this type generalise to a non convex problem of purchasing electricity. Where the classical equivalence breaks down we provide alternative connections to optimal stopping problems. We consider a non convex infinite time horizon SSC problem whose state consists of an uncontrolled diffusion representing a real-va…
The paper studies a class of multidimensional optimal stopping problems with infinite horizon for linear switching diffusions. There are two main novelties in the optimal problems considered: the underlying stochastic process has discontinuous paths and the cost function is not necessarily integrable on the entire time horizon, where the latter is often a key assumption in classical optimal stopping theory for diffusions, cf. [22, Corollary 2.9]. Under relatively mild conditions, we show, for the class of multidimensional optimal stopping problems under consideration, that the first entry time…
We consider a stochastic game of control and stopping specified in terms of a process $X_t=-θΛ_t+W_t$, representing the holdings of Player 1, where $W$ is a Brownian motion, $θ$ is a Bernoulli random variable indicating whether Player 2 is active or not, and $Λ$ is a non-decreasing process representing the accumulated "theft" or "fraud" performed by Player 2 (if active) against Player 1. Player 1 cannot observe $θ$ or $Λ$ directly, but can merely observe the path of the process $X$ and may choose a stopping rule $τ$ to deactivate Player 2 at a cost $M$. Player 1 thus does not know if she is th…
Consider a set of discounted optimal stopping problems for a one-parameter family of objective functions and a fixed diffusion process, started at a fixed point. A standard problem in stochastic control/optimal stopping is to solve for the problem value in this setting. In this article we consider an inverse problem; given the set of problem values for a family of objective functions, we aim to recover the diffusion. Under a natural assumption on the family of objective functions we can characterise existence and uniqueness of a diffusion for which the optimal stopping problems have the specif…
We consider a pair $(X,Y)$ of stochastic processes satisfying the equation $dX=a(X)Y\,dB$ driven by a Brownian motion and study the monotonicity and continuity in $y$ of the value function $v(x,y)=\sup_τE_{x,y}[e^{-qτ}g(X_τ)]$, where the supremum is taken over stopping times with respect to the filtration generated by $(X,Y)$. Our results can successfully be applied to pricing American options where $X$ is the discounted price of an asset while $Y$ is given by a stochastic volatility model such as those proposed by Heston or Hull and White. The main method of proof is based on time-change and …
We study a combined optimal control/stopping problem under a nonlinear expectation ${\cal E}^f$ induced by a BSDE with jumps, in a Markovian framework. The terminal reward function is only supposed to be Borelian. The value function $u$ associated with this problem is generally irregular. We first establish a {\em sub- (resp. super-) optimality principle of dynamic programming} involving its {\em upper- (resp. lower-) semicontinuous envelope} $u^*$ (resp. $u_*$). This result, called {\em weak} dynamic programming principle (DPP), extends that obtained in \cite{BT} in the case of a classical ex…
We consider an investment problem in which an investor performs capital injections to increase the liquidity of a firm for it to maximise profit from market operations. Each time the investor performs an injection, the investor incurs a fixed transaction cost. In addition to maximising their terminal reward, the investor seeks to minimise the risk of loss of their investment (from a possible firm ruin) by exiting the market at some point in time. We show that the problem can be reformulated in terms of a new stochastic differential game of control and stopping in which one of the players modif…
We provide sufficient conditions for the continuity of the free-boundary in a general class of finite-horizon optimal stopping problems arising for instance in finance and economics. The underlying process is a strong solution of one dimensional, time-homogeneous stochastic differential equation (SDE). The proof relies on both analytic and probabilistic arguments and it is based on a contradiction scheme inspired by the maximum principle in partial differential equations (PDE) theory. Mild, local regularity of the coefficients of the SDE and smoothness of the gain function locally at the bound…
We consider Backward Stochastic Differential Equations in a setting where noise is generated by a countable state, continuous time Markov chain, and the terminal value is prescribed at a stopping time. We show that, given sufficient integrability of the stopping time and a growth bound on the terminal value and BSDE driver, these equations admit unique solutions satisfying the same growth bound (up to multiplication by a constant). This holds without assuming that the driver is monotone in y, that is, our results do not require that the terminal value be discounted at some uniform rate. We sho…
We use martingale and stochastic analysis techniques to study a continuous-time optimal stopping problem, in which the decision maker uses a dynamic convex risk measure to evaluate future rewards. We also find a saddle point for an equivalent zero-sum game of control and stopping, between an agent (the "stopper") who chooses the termination time of the game, and an agent (the "controller", or "nature") who selects the probability measure.
We solve a problem of singular stochastic control with discretionary stopping, suggested as an interesting open problem by Karatzas, Ocone, Wang and Zervos (2000), by providing suitable candidates for the moving boundaries in an unsolved parameter range. We proceed by identifying an optimal stopping problem with similar variational inequalities and inspecting its parameter-dependent geometry (in a sense going back to Dynkin (1965)), which reveals a discontinuity not previously exploited. We thus highlight the potential importance of this geometric information in both singular control and param…
We adapt the Stochastic Perron's method in Bayraktar and Sirbu (ArXiv: 1103.0538) to the case of double obstacle problems associated to Dynkin games. We construct, symmetrically, a viscosity sub-solution which dominates the upper value of the game and a viscosity super-solution lying below the lower value of the game. If the double obstacle problem satisfies the viscosity comparison property, then the game has a value which is equal to the unique and continuous viscosity solution. In addition, the optimal strategies of the two players are equal to the first hitting times of the two stopping re…
We introduce the notion of mild supersolution for an obstacle problem in an infinite dimensional Hilbert space. The minimal supersolution of this problem is given in terms of a reflected BSDEs in an infinite dimensional Markovian framework. The results are applied to an optimal control and stopping problem.
Computational Science Initiative（计算科学计划）；Brookhaven National Laboratory（布鲁克海文国家实验室）
This work examines the convergence of stochastic gradient-based optimization algorithms that use early stopping based on a validation function. The form of early stopping we consider is that optimization terminates when the norm of the gradient of a validation function falls below a threshold. We derive conditions that guarantee this stopping rule is well-defined, and provide bounds on the expected number of iterations and gradient evaluations needed to meet this criterion. The guarantee accounts for the distance between the training and validation sets, measured with the Wasserstein distance.…
In this article we continue our investigation of the iterative regularization method for optimization problems based on Bregman distances. The optimization problems are subject to pointwise inequality constraints in $L^2(Ω)$. We provide an estimate for the noise error for perturbed data, which can be used to construct an a priori stopping rule. Furthermore we show how to implement our method with a semi-smooth Newton method using finite elements and present numerical results for the stopping rule.
We consider an optimal stopping problem where a constraint is placed on the distribution of the stopping time. Reformulating the problem in terms of so-called measure-valued martingales allows us to transform the marginal constraint into an initial condition and view the problem as a stochastic control problem; we establish the corresponding dynamic programming principle.
We study the numerical solution of nonlinear partially observed optimal stopping problems. The system state is taken to be a multi-dimensional diffusion and drives the drift of the observation process, which is another multi-dimensional diffusion with correlated noise. Such models where the controller is not fully aware of her environment are of interest in applied probability and financial mathematics. We propose a new approximate numerical algorithm based on the particle filtering and regression Monte Carlo methods. The algorithm maintains a continuous state-space and yields an integrated ap…
We study the optimal dividend problem for a firm's manager who has partial information on the profitability of the firm. The problem is formulated as one of singular stochastic control with partial information on the drift of the underlying process and with absorption. In the Markovian formulation, we have a 2-dimensional degenerate diffusion, whose first component is singularly controlled and it is absorbed as it hits zero. The free boundary problem (FBP) associated to the value function of the control problem is challenging from the analytical point of view due to the interplay of degeneracy…
We analyze an optimal stopping problem with random maturity under a nonlinear expectation with respect to a weakly compact set of mutually singular probabilities $\mathcal{P}$. The maturity is specified as the hitting time to level $0$ of some continuous index process at which the payoff process is even allowed to have a positive jump. When $\mathcal{P}$ is a collection of semimartingale measures, the optimal stopping problem can be viewed as a {\it discretionary} stopping problem for a player who can influence both drift and volatility of the dynamic of underlying stochastic flow.
We study a zero-sum game where the evolution of a spectrally one-sided Levy process is modified by a singular controller and is terminated by the stopper. The singular controller minimizes the expected values of running, controlling and terminal costs while the stopper maximizes them. Using fluctuation theory and scale functions, we derive a saddle point and the value function of the game. Numerical examples under phase-type Levy processes are also given.
We formulate a stochastic game of mean field type where the agents solve optimal stopping problems and interact through the proportion of players that have already stopped. Working with a continuum of agents, typical equilibria become functions of the common noise that all agents are exposed to, whereas idiosyncratic randomness can be eliminated by an Exact Law of Large Numbers. Under a structural monotonicity assumption, we can identify equilibria with solutions of a simple equation involving the distribution function of the idiosyncratic noise. Solvable examples allow us to gain insight into…
In this paper, we first establish the reflected backward stochastic difference equations with finite state (FS-RBSDEs for short). Then we explore the Existence and Uniqueness Theorem as well as the Comparison Theorem by "one step" method. The connections between FS-RBSDEs and optimal stopping time problems are investigated and we also show that the optimal stopping problems with multiple priors under Knightian uncertainty is a special case of our FS-RBSDEs. As a byproduct we develop the general theory of g-martingales in discrete time with finite state including Doob-Mayer Decomposition Theore…
