We propose a numerical method to approximate the value function for the optimal stopping problem of a piecewise deterministic Markov process (PDMP). Our approach is based on quantization of the post jump location---inter-arrival time Markov chain naturally embedded in the PDMP, and path-adapted time discretization grids. It allows us to derive bounds for the convergence rate of the algorithm and to provide a computable $ε$-optimal stopping time. The paper is illustrated by a numerical example.
This paper investigates the random horizon optimal stopping problem for measure-valued piecewise deterministic Markov processes (PDMPs). This is motivated by population dynamics applications, when one wants to monitor some characteristics of the individuals in a small population. The population and its individual characteristics can be represented by a point measure. We first define a PDMP on a space of locally finite measures. Then we define a sequence of random horizon optimal stopping problems for such processes. We prove that the value function of the problems can be obtained by iterating …
In this paper we aim to construct infinite dimensional versions of well established Piecewise Deterministic Monte Carlo methods, such as the Bouncy Particle Sampler, the Zig-Zag Sampler and the Boomerang Sampler. In order to do so we provide an abstract infinite-dimensional framework for Piecewise Deterministic Markov Processes (PDMPs) with unbounded event intensities. We further develop exponential convergence to equilibrium of the infinite dimensional Boomerang Sampler, using hypocoercivity techniques. Furthermore we establish how the infinite dimensional Boomerang Sampler admits a finite di…
This paper deals with the general discounted impulse control problem of a piecewise deterministic Markov process. We investigate a new family of epsilon-optimal strategies. The construction of such strategies is explicit and only necessitates the previous knowledge of the cost of the no-impulse strategy. In particular, it does not require the resolution of auxiliary optimal stopping problem or the computation of the value function at each point of the state space. This approach is based on the iteration of a single-jump-or-intervention operator associated to the piecewise deterministic Markov …
We consider piecewise deterministic Markov processes with degenerate transition kernels of the "house-of-cards"-type. We use a splitting scheme based on jump times to prove the absolute continuity, as well as some regularity, of the invariant measure of the process. Finally, we obtain finer results on the regularity of the one-dimensional marginals of the invariant measure, using integration by parts with respect to the jump times.
The purpose of this paper is to study a Markovian metapopulation model on a directed graph with edge-supported transfers and deterministic intra-nodal population dynamics. We first state tractable stability conditions for two typical frameworks motivated by applications: constant jump rates with multiplicative transfer amplitudes, and coercive jump rates with unitary transfers. More general criteria for boundedness, petiteness and ergodicity are then given.
In this paper, we are interested in the exact simulation of a class of Piecewise Deterministic Markov Processes (PDMP). We show how to perform efficient thinning algorithms depending on the jump rate bound. For different types of jump rate bounds, we compare theoretically the mean number of generated (total) jump times and we compare numerically the simulation times. We use thinning algorithms on Hodgkin-Huxley models with Markovian ion channels dynamic to illustrate the results.
Given an energy potential on the Euclidian space, a piecewise deterministic Markov process is designed to sample the corresponding Gibbs measure. In dimension one an Eyring-Kramers formula is obtained for the exit time of the domain of a local minimum at low temperature, and a necessary and sufficient condition is given on the cooling schedule in a simulated annealing algorithm to ensure the process converges to the set of global minima. This condition is similar to the classical one for diffusions and involves the critical depth of the potential. In higher dimension a non optimal sufficient c…
In the first part of this paper we study approximations of trajectories of Piecewise Deter-ministic Processes (PDP) when the flow is not explicit by the thinning method. We also establish a strong error estimate for PDPs as well as a weak error expansion for Piecewise Deterministic Markov Processes (PDMP). These estimates are the building blocks of the Multilevel Monte Carlo (MLMC) method which we study in the second part. The coupling required by MLMC is based on the thinning procedure. In the third part we apply these results to a 2-dimensional Morris-Lecar model with stochastic ion channels…
In this paper we define an infinite-dimensional controlled piecewise deterministic Markov process (PDMP) and we study an optimal control problem with finite time horizon and unbounded cost. This process is a coupling between a continuous time Markov Chain and a set of semilinear parabolic partial differential equations, both processes depending on the control. We apply dynamic programming to the embedded Markov decision process to obtain existence of optimal relaxed controls and we give some sufficient conditions ensuring the existence of an optimal ordinary control. This study, which constitu…
We consider a general piecewise deterministic Markov process (PDMP) $X=\{X_t\}_{t\geqslant 0}$ with measure-valued generator $\mathcal{A}$, for which the conditional distribution function of the inter-occurrence time is not necessarily absolutely continuous. A general form of the exponential martingales is presented as $$M^f_t=\frac{f(X_t)}{f(X_0)}\left[\mathrm{Sexp}\left(\int_{(0,t]}\frac{\mathrm{d}L(\mathcal{A}f)_s}{f(X_{s-})}\right)\right]^{-1}.$$ Using this exponential martingale as a likelihood ratio process, we define a new probability measure. It is shown that the original process remai…
We solve optimal stopping problems for an oscillating Brownian motion, i.e. a diffusion with positive piecewise constant volatility changing at the point $x=0$. Let $σ_1$ and $σ_2$ denote the volatilities on the negative and positive half-lines, respectively. Our main result is that continuation region of the optimal stopping problem with reward $((1+x)^+)^2$ is disconnected, if and only if $σ_1^2<σ_2^2<2σ_1^2$. Based on the fact that the skew Brownian motion in natural scale is an oscillating Brownian motion, the obtained results are translated into corresponding results for the skew Brownian…
We consider multiscale stochastic spatial gene networks involving chemical reactions and diffusions. The model is Markovian and the transitions are driven by Poisson random clocks. We consider a case where there are two different spatial scales: a microscopic one with fast dynamic and a macroscopic one with slow dynamic. At the microscopic level, the species are abundant and for the large population limit a partial differential equation (PDE) is obtained. On the contrary at the macroscopic level, the species are not abundant and their dynamic remains governed by jump processes. It results that…
In Turitsyn, Chertkov, Vucelja (2011) a non-reversible Markov Chain Monte Carlo (MCMC) method on an augmented state space was introduced, here referred to as Lifted Metropolis-Hastings (LMH). A scaling limit of the magnetization process in the Curie-Weiss model is derived for LMH, as well as for Metropolis-Hastings (MH). The required jump rate in the high (supercritical) temperature regime equals $n^{1/2}$ for LMH, which should be compared to $n$ for MH. At the critical temperature the required jump rate equals $n^{3/4}$ for LMH and $n^{3/2}$ for MH, in agreement with experimental results of T…
A piecewise-deterministic Markov process, specified by random jumps and switching semi-flows, as well as the associated Markov chain given by its post-jump locations, are investigated in this paper. The existence of an exponentially attracting invariant measure and the strong law of large numbers are proven for the chain. Further, a one-to-one correspondence between invariant measures for the chain and invariant measures for the continuous-time process is established. This result, together with the aforementioned ergodic properties of the discrete-time model, is used to derive the strong law o…
We investigate a piecewise-deterministic Markov process, evolving on a Polish metric space, whose deterministic behaviour between random jumps is governed by some semi-flow, and any state right after the jump is attained by a randomly selected continuous transformation. It is assumed that the jumps appear at random moments, which coincide with the jump times of a Poisson process with intensity $λ$. The model of this type, although in a more general version, was examined in our previous papers, where we have shown, among others, that the Markov process under consideration possesses a unique inv…
Piecewise deterministic Markov processes are an important new tool in the design of Markov Chain Monte Carlo algorithms. Two examples of fundamental importance are the Bouncy Particle Sampler (BPS) and the Zig-Zag process (ZZ). In this paper scaling limits for both algorithms are determined. Here the dimensionality of the space tends towards infinity and the target distribution is the multivariate standard normal distribution. For several quantities of interest (angular momentum, first coordinate, and negative log-density) the scaling limits show qualitatively very different and rich behaviour…
We consider a piecewise-deterministic Markov process (PDMP) with general conditional distribution of inter-occurrence time, which is called a general PDMP here. Our purpose is to establish the theory of measure-valued generator for general PDMPs. The additive functional of a semi-dynamic system (SDS) is introduced firstly, which presents us an analytic tool for the whole paper. The additive functionals of a general PDMP are represented in terms of additive functionals of the SDS. The necessary and sufficient conditions of being a local martingale or a special semimartingale for them are given.…
We investigate piecewise deterministic Markov processes (PDMP), where the deterministic dynamics follows a scalar conservation law and random jumps in the system are characterized by changes in the flux function. We show under which assumptions we can guarantee the existence of a PDMP and conclude bounded variation estimates for sample paths. Finally, we apply this dynamics to a production and traffic model and use this framework to incorporate the well-known scattering of flux functions observed in data sets.
The law of the iterated logarithm for a piecewise deterministic Markov process assured by the properties of the Markov chain given by its post-jump locations
For a class of piecewise deterministic Markov processes, the supports of the invariant measures are characterized. This is based on the analysis of controllability properties of an associated deterministic control system. Its invariant control sets determine the supports.
We present a short introduction into the framework of piecewise deterministic Markov processes. We illustrate the abstract mathematical setting with a series of examples related to dispersal of biological systems, cell cycle models, gene expression, physiologically structured populations, as well as neural activity. General results concerning asymptotic properties of stochastic semigroups induced by such Markov processes are applied to specific examples.
In this paper, a class of piecewise deterministic Markov processes with underlying fast dynamic is studied. Using a "penalty method" , an averaging result is obtained when the underlying dynamic is infinitely accelerated. The features of the averaged process, which is still a piecewise deterministic Markov process, are fully described.
In this paper, we consider a piecewise deterministic Markov process (PDMP), with known flow and deterministic transition measure, and unknown jump rate $λ$. To estimate nonparametrically the jump rate, we first construct an adaptive estimator of the stationary density, then we derive a quotient estimator $\hatλ_n$ of $λ$. We provide uniform bounds for the risk of these estimators, and prove that the estimator of the jump rate is nearly minimax (up to a $\ln^2(n)$ factor). Simulations illustrate the behavior of our estimator.
We extend the hypocoercivity framework for piecewise-deterministic Markov process (PDMP) Monte Carlo established in [Andrieu et. al. (2018)] to heavy-tailed target distributions, which exhibit subgeometric rates of convergence to equilibrium. We make use of weak Poincaré inequalities, as developed in the work of [Grothaus and Wang (2019)], the ideas of which we adapt to the PDMPs of interest. On the way we report largely potential-independent approaches to bounding explicitly solutions of the Poisson equation of the Langevin diffusion and its first and second derivatives, required here to cont…
Piecewise Deterministic Markov Processes (PDMPs) are studied in a general framework. First, different constructions are proven to be equivalent. Second, we introduce a coupling between two PDMPs following the same differential flow which implies quantitative bounds on the total variation between the marginal distributions of the two processes. Finally two results are established regarding the invariant measures of PDMPs. A practical condition to show that a probability measure is invariant for the associated PDMP semi-group is presented. In a second time, a bound on the invariant probability m…
We establish $L^2$-exponential convergence rate for three popular piecewise deterministic Markov processes for sampling: the randomized Hamiltonian Monte Carlo method, the zigzag process, and the bouncy particle sampler. Our analysis is based on a variational framework for hypocoercivity, which combines a Poincaré-type inequality in time-augmented state space and a standard $L^2$ energy estimate. Our analysis provides explicit convergence rate estimates, which are more quantitative than existing results.
We present recent results on Piecewise Deterministic Markov Processes (PDMPs), involved in biological modeling. PDMPs, first introduced in the probabilistic literature by Davis (1984), are a very general class of Markov processes and are being increasingly popular in biological applications. They also give new interesting challenges from the theoretical point of view. We give here different examples on the long time behavior of switching Markov models applied to population dynamics, on uniform sampling in general branching models applied to structured population dynamic, on time scale separati…
In this work, we give sharp asymptotic equivalents in the small temperature regime of the smallest eigenvalues of the generator of some piecewise deterministic Markov processes (including the ZigZag process and the Bouncy Particle Sampler process) with refreshment rate $α$ on the one-dimensional torus T. These asymptotic equivalents are usually called Eyring-Kramers type formulas in the literature. The case when the refreshment rate $α$ vanishes on T is also considered.
The Bouncy Particle sampler (BPS) and the Zig-Zag sampler (ZZS) are continuous time, non-reversible Monte Carlo methods based on piecewise deterministic Markov processes. Experiments show that the speed of convergence of these samplers can be affected by the shape of the target distribution, as for instance in the case of anisotropic targets. We propose an adaptive scheme that iteratively learns all or part of the covariance matrix of the target and takes advantage of the obtained information to modify the underlying process with the aim of increasing the speed of convergence. Moreover, we def…
In this paper, we analyse piecewise deterministic Markov processes, as introduced in Davis (1984). Many models in insurance mathematics can be formulated in terms of the general concept of piecewise deterministic Markov processes. In this context, one is interested in computing certain quantities of interest such as the probability of ruin of an insurance company, or the insurance company's value, defined as the expected discounted future dividend payments until the time of ruin. Instead of explicitly solving the integro-(partial) differential equation related to the quantity of interest consi…
Exponential ergodicity in the bounded-Lipschitz distance for a subclass of piecewise-deterministic Markov processes with random switching between flows
In piecewise-deterministic Markov processes (PDMPs) the state of a finite-dimensional system evolves continuously, but the evolutive equation may change randomly as a result of discrete switches. A running cost is integrated along the corresponding piecewise-deterministic trajectory up to the termination to produce the cumulative cost of the process. We address three natural questions related to uncertainty in cumulative cost of PDMP models: (1) how to compute the Cumulative Distribution Function (CDF) of the cumulative cost when the switching rates are fully known; (2) how to accurately bound…
