In this paper, we consider the optimal stopping problem on semi-Markov processes (SMPs) with finite horizon, and aim to establish the existence and computation of optimal stopping times. To achieve the goal, we first develop the main results of finite horizon semi-Markov decision processes (SMDPs) to the case with additional terminal costs, introduce an explicit construction of SMDPs, and prove the equivalence between the optimal stopping problems on SMPs and SMDPs. Then, using the equivalence and the results on SMDPs developed here, we not only show the existence of optimal stopping time of SMPs, but also provide an algorithm for computing optimal stopping time on SMPs. Moreover, we show that the optimal and "-optimal stopping time can be characterized by the hitting time of some special sets, respectively.
The paper is concerned with a variant of the continuous-time finite state Markov game of control and stopping where both players can affect transition rates, while only one player can choose a stopping time. We use the dynamic programming principle and reduce this problem to a system of ODEs with unilateral constraints. This system plays the role of the Bellman equation. We show that its solution provides the optimal strategies of the players. Additionally, we prove the existence and uniqueness theorem for the deduced system of ODEs with unilateral constraints.
Previous authors have considered optimal stopping problems driven by the running maximum of a spectrally negative Lévy process $X$, as well as of a one-dimensional diffusion. Many of the aforementioned results are either implicitly or explicitly dependent on Peskir's maximality principle. In this article, we are interested in understanding how some of the main ideas from these previous works can be brought into the setting of problems driven by the maximum of a class of Markov additive processes (more precisely Markov modulated Lévy processes). Similarly to previous works in the Lévy setting, …
This paper attempts to study the optimal stopping time for semi-Markov processes (SMPs) under the discount optimization criteria with unbounded cost rates. In our work, we introduce an explicit construction of the equivalent semi-Markov decision processes (SMDPs). The equivalence is embodied in the value functions of SMPs and SMDPs, that is, every stopping time of SMPs can induce a policy of SMDPs such that the value functions are equal, and vice versa. The existence of the optimal stopping time of SMPs is proved by this equivalence relation. Next, we give the optimality equation of the value …
Standard Markovian optimal stopping problems are consistent in the sense that the first entrance time into the stopping set is optimal for each initial state of the process. Clearly, the usual concept of optimality cannot in a straightforward way be applied to non-standard stopping problems without this time-consistent structure. This paper is devoted to the solution of time-inconsistent stopping problems with the reward depending on the initial state using an adaptation of Strotz's consistent planning. More precisely, we give a precise equilibrium definition --- of the type subgame perfect Na…
In this paper, we propose an extension of the forward improvement iteration algorithm, originally introduced in Irle (2006) and recently reconsidered in Miclo and Villeneuve (2021). The main new ingredient is a flexible window parameter describing the look-ahead distance in the improvement step. We consider the framework of a Markovian optimal stopping problem in discrete time with random discounting and infinite time horizon. We prove convergence and show that the additional flexibility may significantly reduce the runtime.
In this paper we consider stopping problems for continuous-time Markov chains under a general risk-sensitive optimization criterion for problems with finite and infinite time horizon. More precisely our aim is to maximize the certainty equivalent of the stopping reward minus cost over the time horizon. We derive optimality equations for the value functions and prove the existence of optimal stopping times. The exponential utility is treated as a special case. In contrast to risk-neutral stopping problems it may be optimal to stop between jumps of the Markov chain. We briefly discuss the influe…
We study a model of two-player, zero-sum, stopping games with asymmetric information. We assume that the payoff depends on two continuous-time Markov chains (X, Y), where X is only observed by player 1 and Y only by player 2, implying that the players have access to stopping times with respect to different filtrations. We show the existence of a value in mixed stopping times and provide a variational characterization for the value as a function of the initial distribution of the Markov chains. We also prove a verification theorem for optimal stopping rules which allows to construct optimal sto…
We consider optimal stopping problems with finite-time horizon and state-dependent discounting. The underlying process is a one-dimensional linear diffusion and the gain function is time-homogeneous and difference of two convex functions. Under mild technical assumptions with local nature we prove fine regularity properties of the optimal stopping boundary including its continuity and strict monotonicity. The latter was never proven with probabilistic arguments. We also show that atoms in the signed measure associated with the second order spatial derivative of the gain function induce geometr…
Suppose that $\mathcal C$ is a finite collection of patterns. Observe a Markov chain until one of the patterns in $\mathcal C$ occurs as a run. This time is denoted by $τ$. In this paper, we aim to give an easy way to calculate the mean waiting time $E(τ)$ and the stopping probabilities $P(τ=τ_A)$ with $A\in\mathcal C$, where $τ_A$ is the waiting time until the pattern $A$ appears as a run.
This paper is concerned with the solution of the optimal stopping problem associated to the valuation of Perpetual American options driven by continuous time Markov chains. We introduce a new dynamic approach for the numerical pricing of this type of American options where the main idea is to build a monotone sequence of almost excessive functions that are associated to hitting times of explicit sets. Under minimal assumptions about the payoff and the Markov chain, we prove that the value function of an American option is characterized by the limit of this monotone sequence.
The paper studies a class of multidimensional optimal stopping problems with infinite horizon for linear switching diffusions. There are two main novelties in the optimal problems considered: the underlying stochastic process has discontinuous paths and the cost function is not necessarily integrable on the entire time horizon, where the latter is often a key assumption in classical optimal stopping theory for diffusions, cf. [22, Corollary 2.9]. Under relatively mild conditions, we show, for the class of multidimensional optimal stopping problems under consideration, that the first entry time…
For classical finite time horizon stopping problems driven by a Brownian motion \[V(t,x) = \sup_{t\leqτ\leq0}E_{(t,x)}[g(τ,W_τ)],\] we derive a new class of Fredholm type integral equations for the stopping set. For large problem classes of interest, we show by analytical arguments that the equation uniquely characterizes the stopping boundary of the problem. Regardless of the uniqueness, we use the representation to rigorously find the limit behavior of the stopping boundary close to the terminal time. Interestingly, it turns out that the leading-order coefficient is universal for wide classe…
In this paper we consider an infinite time horizon risk-sensitive optimal stopping problem for a Feller--Markov process with an unbounded terminal cost function. We show that in the unbounded case an associated Bellman equation may have multiple solutions and we give a probabilistic interpretation for the minimal and the maximal one. Also, we show how to approximate them using finite time horizon problems. The analysis, covering both discrete and continuous time case, is supported with illustrative examples.
This paper investigates the random horizon optimal stopping problem for measure-valued piecewise deterministic Markov processes (PDMPs). This is motivated by population dynamics applications, when one wants to monitor some characteristics of the individuals in a small population. The population and its individual characteristics can be represented by a point measure. We first define a PDMP on a space of locally finite measures. Then we define a sequence of random horizon optimal stopping problems for such processes. We prove that the value function of the problems can be obtained by iterating …
In this paper, we present a discrete-type approximation scheme to solve continuous-time optimal stopping problems based on fully non-Markovian continuous processes adapted to the Brownian motion filtration. The approximations satisfy suitable variational inequalities which allow us to construct $ε$-optimal stopping times and optimal values in full generality. Explicit rates of convergence are presented for optimal values based on reward functionals of path-dependent SDEs driven by fractional Brownian motion. In particular, the methodology allows us to design concrete Monte-Carlo schemes for no…
We propose a method based on continuous time Markov chain approximation to compute the distribution of Parisian stopping times and price Parisian options under general one-dimensional Markov processes. We prove the convergence of the method under a general setting and obtain sharp estimate of the convergence rate for diffusion models. Our theoretical analysis reveals how to design the grid of the CTMC to achieve faster convergence. Numerical experiments are conducted to demonstrate the accuracy and efficiency of our method for both diffusion and jump models. To show the versality of our approa…
In this paper we consider discrete and continuous time risk sensitive optimal stopping problem. Using suitable properties of the underlying Feller-Markov process we prove continuity of the optimal stopping value function and provide formula for the optimal stopping policy. Next, we show how to link continuous time framework with its discrete time analogue. By considering suitable approximations, we obtain uniform convergence of the corresponding value functions.
Optimal Stopping for Strong Markov Processes: Explicit solutions and verification theorems for diffusions, multidimensional diffusions, and jump-processes
In this paper, we present a Longstaff-Schwartz-type algorithm for optimal stopping time problems based on the Brownian motion filtration. The algorithm is based on Leão, Ohashi and Russo and, in contrast to previous works, our methodology applies to optimal stopping problems for fully non-Markovian and non-semimartingale state processes such as functionals of path-dependent stochastic differential equations and fractional Brownian motions. Based on statistical learning theory techniques, we provide overall error estimates in terms of concrete approximation architecture spaces with finite Vapni…
We provide a new probabilistic proof of the connection between Rost's solution of the Skorokhod embedding problem and a suitable family of optimal stopping problems for Brownian motion with finite time-horizon. In particular we use stochastic calculus to show that the time reversal of the optimal stopping sets for such problems forms the so-called Rost's reversed barrier.
We study the stochastic control-stopping problem when the data are of polynomial growth. The approach is based on backward stochastic dierential equations (BSDEs for short). The problem turns into the study of a specic reected BSDE with a stochastic Lipschitz coecient for which we show existence and uniqueness of the solution. We then establish its relationship with the value function of the control-stopping problem. The optimal strategy is exhibited. Finally in the Markovian framework we prove that the value function is the unique viscosity solution of the associated Hamilton-Jacobi-Bellman e…
We provide general conditions ensuring that the value functions of some nonlinear stopping problems with finite horizon converge to the value functions of the corresponding problems with infinite horizon. Our result can be formulated as result on stability, with respect to time horizon, of nonlinear $f$-expectations. We also study the rate of convergence. Many examples are given to illustrate our results. They include the analysis of time asymptotics of the fair prices of American options in a multidimensional exponential Lévy model.
In this paper we consider stopping problems with partial observation under a general risk-sensitive optimization criterion for problems with finite and infinite time horizon. Our aim is to maximize the certainty equivalent of the stopping reward. We develop a general theory and discuss the Bayesian risk-sensitive house selling problem as a special example. In particular we are able to study the influence of the attitude towards risk of the decision maker on the optimal stopping rule.
We develop a theory of optimal stopping problems under G-expectation framework. We first define a new kind of random times, called G-stopping times, which is suitable for this problem. For the discrete time case with finite horizon, the value function is defined backwardly and we show that it is the smallest G-supermartingale dominating the payoff process and the optimal stopping time exists. Then we extend this result both to the infinite horizon and to the continuous time case. We also establish the relation between the value function and solution of reflected BSDE driven by G-Brownian motio…
We consider a zero-sum stochastic game for continuous-time Markov chain with countable state space and unbounded transition and pay-off rates. The additional feature of the game is that the controllers together with taking actions are also allowed to stop the process. Under suitable hypothesis we show that the game has a value and it is the unique solution of certain dynamic programming inequalities with bilateral constraints. In the process we also prescribe a saddle point equilibrium.
Consider a real hyperplane arrangement and let $\mathcal{C}$ denote the occurring chambers. Bidigare, Hanlon and Rockmore introduced a Markov chain on $\mathcal{C}$ which is a generalization of some card shuffling models used in computer science, biology and card games. This paper introduces strong stationary arguments for this Markov chain, which provide explicit bounds for the separation distance.
A diffusion spider is a strong Markov process with continuous paths taking values on a graph with one vertex and a finite number of edges (of infinite length). An example is Walsh's Brownian spider where the process on each edge behaves as Brownian motion. We calculate the density of the resolvent kernel in terms of the characteristics of the underlying diffusion. Excessive functions are studied via the Martin boundary theory. The main result is an explicit expression for the representing measure of a given excessive function. These results are used to solve optimal stopping problems for diffu…
