Sourav Chatterjee, Persi Diaconis
We show that the convergence of finite state space Markov chains to stationarity can often be considerably speeded up by alternating every step of the chain with a deterministic move. Under fairly general conditions, we show that not only do such schemes exist, they are numerous.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
继续探索
We consider piecewise deterministic Markov processes with degenerate transition kernels of the "house-of-cards"-type. We use a splitting scheme based on jump times to prove the absolute continuity, as well as some regularity, of the invariant measure of the process. Finally, we obtain finer results on the regularity of the one-dimensional marginals of the invariant measure, using integration by parts with respect to the jump times.
In this paper, we are interested in the exact simulation of a class of Piecewise Deterministic Markov Processes (PDMP). We show how to perform efficient thinning algorithms depending on the jump rate bound. For different types of jump rate bounds, we compare theoretically the mean number of generated (total) jump times and we compare numerically the simulation times. We use thinning algorithms on Hodgkin-Huxley models with Markovian ion channels dynamic to illustrate the results.
On the Existence and Uniqueness of Stationary Distributions for Some Piecewise Deterministic Markov Processes With State-Dependent Jump Intensity
本文研究一类分段确定性马尔可夫过程，证明其平稳分布与跳跃后位置马尔可夫链的平稳分布一一对应，并据此为随机迭代函数系统情形建立平稳分布存在唯一性判据。
We investigate a piecewise-deterministic Markov process, evolving on a Polish metric space, whose deterministic behaviour between random jumps is governed by some semi-flow, and any state right after the jump is attained by a randomly selected continuous transformation. It is assumed that the jumps appear at random moments, which coincide with the jump times of a Poisson process with intensity $λ$. The model of this type, although in a more general version, was examined in our previous papers, where we have shown, among others, that the Markov process under consideration possesses a unique inv…
We study the time reversal of a general PDMP. The time reversed process is defined as $X_{(T-t)-}$, where $T$ is some given time and $X_t$ is a stationary PDMP. We obtain the parameters of the reversed process, like the jump intensity and the jump measure.
The law of the iterated logarithm for a piecewise deterministic Markov process assured by the properties of the Markov chain given by its post-jump locations
本文研究半流由 Poisson 跳变切换的分段确定性 Markov 过程，借助其跳后 Markov 链的已有遍历性质证明连续时间过程的重对数律，并指出其可用于基因表达随机动力学建模。
Kigami showed that a transient random walk on a deterministic infinite tree $T$ induces its trace process on the Martin boundary of $T$. In this paper, we will deal with trace processes on Martin boundaries of random trees instead of deterministic ones, and prove short time log-asymptotic of on-diagonal heat kernel estimates and estimates of mean displacements.
The purpose of this paper is to study a Markovian metapopulation model on a directed graph with edge-supported transfers and deterministic intra-nodal population dynamics. We first state tractable stability conditions for two typical frameworks motivated by applications: constant jump rates with multiplicative transfer amplitudes, and coercive jump rates with unitary transfers. More general criteria for boundedness, petiteness and ergodicity are then given.
We consider a piecewise-deterministic Markov process governed by a jump intensity function, a rate function that determines the behaviour between jumps, and a stochastic kernel describing the conditional distribution of jump sizes. We study the point process of upcrossings of a level $b$ by the Markov process. Our main result shows that, under a suitable scaling $ν(b)$, the point process converges, as $b$ tends to infinity, weakly to a geometrically compound Poisson process. We also prove a version of Rice's formula relating the stationary density of the process to level crossing intensities. …
In this paper, we consider a piecewise deterministic Markov process (PDMP), with known flow and deterministic transition measure, and unknown jump rate $λ$. To estimate nonparametrically the jump rate, we first construct an adaptive estimator of the stationary density, then we derive a quotient estimator $\hatλ_n$ of $λ$. We provide uniform bounds for the risk of these estimators, and prove that the estimator of the jump rate is nearly minimax (up to a $\ln^2(n)$ factor). Simulations illustrate the behavior of our estimator.
We study the asymptotic behavior of multiscale stochastic gene networks using weak limits of Markov jump processes. Depending on the time and concentration scales of the system we distinguish four types of limits: continuous piecewise deterministic processes (PDP) with switching, PDP with jumps in the continuous variables, averaged PDP, and PDP with singular switching. We justify rigorously the convergence for the four types of limits. The convergence results can be used to simplify the stochastic dynamics of gene network models arising in molecular biology.
On absolute continuity of invariant measures associated with a piecewise-deterministic Markov processes with random switching between flows
本文研究了生物模型中分段确定性 Markov 过程平稳分布的绝对连续性，该过程由 Poisson 过程跳跃和连续半流切换组成，提出了一组可验证条件，证明其不变分布具有关于 Lebesgue 测度的密度。
Many results in stochastic analysis and mathematical finance involve local martingales. However, specific examples of strict local martingales are rare and analytically often rather unhandy. We study local martingales that follow a given deterministic function up to a random time $γ$ at which they jump and stay constant afterwards. The (local) martingale properties of these single jump local martingales are characterised in terms of conditions on the input parameters. This classification allows an easy construction of strict local martingales, uniformly integrable martingales that are not in $…
We study the Markov chain on $\mathbf{F}_p$ obtained by applying a function $f$ and adding $\pmγ$ with equal probability. When $f$ is a linear function, this is the well-studied Chung--Diaconis--Graham process. We consider two cases: when $f$ is the extension of a rational function which is bijective, and when $f(x)=x^2$. In the latter case, the stationary distribution is not uniform and we characterize it when $p=3\pmod{4}$. In both cases, we give an almost linear bound on the mixing time, showing that the deterministic function dramatically speeds up mixing. The proofs involve establishing b…
Optimal Fluctuations for Discrete-time Markov Jump Processes
本文研究了离散时间马尔可夫跳跃过程中噪声诱导的大波动现象，通过大偏差理论和时间反转概念，揭示了最佳路径与特定概率分布时间反转之间的关系，展示了稀有随机事件如何产生确定性机制。
We treat the class of universal Markov processes on the d-dimensional Euklidean space which do not depend on random. For these, as well as for several subclasses, we prove criteria whether a function f, defined on the positive half-line, can be a path of a process in the respective class. This is useful in particular in the construction of (counter-)examples. Furthermore we characterize the processes of this kind, which are homogeneous in space and time. The semimartingale property is characterized in terms of the jumps of a one-dimensional deterministic Markov process. We emphasize the differ…
Let $X$ be a linear diffusion taking values in $(\ell,r)$ and consider the standard Euler scheme to compute an approximation to $\mathbb{E}[g(X_T)\mathbf{1}_{[T<ζ]}]$ for a given function $g$ and a deterministic $T$, where $ζ=\inf\{t\geq 0: X_t \notin (\ell,r)\}$. It is well-known since \cite{GobetKilled} that the presence of killing introduces a loss of accuracy and reduces the weak convergence rate to $1/\sqrt{N}$ with $N$ being the number of discretisatons. We introduce a drift-implicit Euler method to bring the convergence rate back to $1/N$, i.e. the optimal rate in the absence of killing…
In this paper, we consider the generalized Hodgkin-Huxley model introduced by Austin in \cite{Austin}. This model describes the propagation of an action potential along the axon of a neuron at the scale of ion channels. Mathematically, this model is a fully-coupled Piecewise Deterministic Markov Process (PDMP) in infinite dimension. We introduce two time scales in this model in considering that some ion channels open and close at faster jump rates than others. We perform a slow-fast analysis of this model and prove that asymptotically this two time scales model reduces to the so called average…
For a continuous-time Markov process, we characterize the law of the first jump location when started from an arbitrary initial distribution, in terms of the invariant distribution of an auxiliary Markov process. This could be of interest in the burgeoning fields of piecewise-deterministic Markov chain Monte Carlo methods and quasi-stationary Monte Carlo methods.
In Turitsyn, Chertkov, Vucelja (2011) a non-reversible Markov Chain Monte Carlo (MCMC) method on an augmented state space was introduced, here referred to as Lifted Metropolis-Hastings (LMH). A scaling limit of the magnetization process in the Curie-Weiss model is derived for LMH, as well as for Metropolis-Hastings (MH). The required jump rate in the high (supercritical) temperature regime equals $n^{1/2}$ for LMH, which should be compared to $n$ for MH. At the critical temperature the required jump rate equals $n^{3/4}$ for LMH and $n^{3/2}$ for MH, in agreement with experimental results of T…
A piecewise-deterministic Markov process, specified by random jumps and switching semi-flows, as well as the associated Markov chain given by its post-jump locations, are investigated in this paper. The existence of an exponentially attracting invariant measure and the strong law of large numbers are proven for the chain. Further, a one-to-one correspondence between invariant measures for the chain and invariant measures for the continuous-time process is established. This result, together with the aforementioned ergodic properties of the discrete-time model, is used to derive the strong law o…
We cosider random dynamical systems with randomly chosen jumps. The choice of deterministic dynamical system and jumps depends on a position. We proove the existence of an exponentially attractive invariant measure and the strong law of large numbers.
Markov branching processes with disasters: extinction, survival and duality to p-jump processes
本文通过建立 p-跳跃过程在单位区间上的极限定理，并利用概率生成函数对偶性，研究带二项灾难的分支过程的生存概率极限，进而推广到时间非齐次生灭过程和带 p-跳跃的连续状态分支过程。
Mean field analysis of interacting network model with jumps
本文研究带跳的 n 粒子平均场相互作用系统，证明其经验测度收敛到 McKean-Vlasov 方程的解，并给出混沌传播和收敛速度估计。
This paper deals with the long run average continuous control problem of piecewise deterministic Markov processes (PDMP's) taking values in a general Borel space and with compact action space depending on the state variable. The control variable acts on the jump rate and transition measure of the PDMP, and the running and boundary costs are assumed to be positive but not necessarily bounded. Our first main result is to obtain an optimality equation for the long run average cost in terms of a discrete-time optimality equation related to the embedded Markov chain given by the post-jump location …
We consider Piecewise Deterministic Markov Processes (PDMPs) with a finite set of discrete states. In the regime of fast jumps between discrete states, we prove a law of large number and a large deviation principle. In the regime of fast and slow jumps, we analyze a coarse-grained process associated to the original one and prove its convergence to a new PDMP with effective force fields and jump rates. In all the above cases, the continuous variables evolve slowly according to ODEs. Finally, we discuss some applications related to the mechanochemical cycle of macromolecules, including strained-…
Exponential ergodicity in the bounded-Lipschitz distance for a subclass of piecewise-deterministic Markov processes with random switching between flows
本文研究了一类在流之间随机切换的分段确定性马尔可夫过程，提出了仅依赖流性质和跳跃后位置转移律的可验证条件，以保证其在有界Lipschitz距离下的指数遍历性，并为适用于生物模型的特例建立了简单判据。
In this paper we study a particular class of Piecewise deterministic Markov processes (PDMP's) which are semi-stochastic catastrophe versions of deterministic population growth models. In between successive jumps the process follows a flow describing deterministic population growth. Moreover, at random jump times, governed by state-dependent rates, the size of the population shrinks by a random amount of its current size, an event possibly leading to instantaneous local (or total) extinction. A special separable shrinkage transition kernel is investigated in more detail, including the case of …
Reflected Mckean-Vlasov stochastic differential equations with jumps in time-dependent domains
本文研究时间依赖凸区域中带法向反射的McKean-Vlasov随机微分方程，证明了解的存在唯一性、稳定性及混沌传播性质。
We consider a discrete time simple symmetric random walk on Z^d, d>=1, where the path of the walk is perturbed by inserting deterministic jumps. We show that for any time n and any deterministic jumps that we insert, the expected number of sites visited by the perturbed random walk up to time n is always larger than or equal to that for the unperturbed walk. This intriguing problem arises from the study of a particle among a Poisson system of moving traps with sub-diffusive trap motion. In particular, our result implies a variant of the Pascal principle, which asserts that among all determinis…
This paper deals with the general discounted impulse control problem of a piecewise deterministic Markov process. We investigate a new family of epsilon-optimal strategies. The construction of such strategies is explicit and only necessitates the previous knowledge of the cost of the no-impulse strategy. In particular, it does not require the resolution of auxiliary optimal stopping problem or the computation of the value function at each point of the state space. This approach is based on the iteration of a single-jump-or-intervention operator associated to the piecewise deterministic Markov …
The purpose of this paper is to consider the exit-time problem for a finite-range Markov jump process, i.e, the distance the particle can jump is bounded independent of its location. Such jump diffusions are expedient models for anomalous transport exhibiting super-diffusion or nonstandard normal diffusion. We refer to the associated deterministic equation as a volume-constrained nonlocal diffusion equation. The volume constraint is the nonlocal analogue of a boundary condition necessary to demonstrate that the nonlocal diffusion equation is well-posed and is consistent with the jump process. …
It is shown that piecewise deterministic dissipative quantum dynamics in a vector space with indefinite metric can lead to well defined, positive probabilities. The case of quantum jumps on the Poincar'e disk is studied in details, including results of numerical simulations of quantum fractals. Keywords: iterated function systems, dynamical semigroups, EEQT, indefinite metric, quantum jumps, quantum measurement, non-commuting observables, fractals, Poincare disk
This paper deals with the optimal stopping problem under partial observation for piecewise-deterministic Markov processes. We first obtain a recursive formulation of the optimal filter process and derive the dynamic programming equation of the partially observed optimal stopping problem. Then, we propose a numerical method, based on the quantization of the discrete-time filter process and the inter-jump times, to approximate the value function and to compute an actual $ε$-optimal stopping time. We prove the convergence of the algorithms and bound the rates of convergence.
The standard practice in modeling dynamics and optimal control of a large population, ensemble, multi-agent system represented by it's continuum density, is to model individual decision making using local feedback information. In comparison to a closed-loop optimal control scheme, an open-loop strategy, in which a centralized controller broadcasts identical control signals to the ensemble of agents, mitigates the computational and infrastructure requirements for such systems. This work considers the open-loop, deterministic and optimal control synthesis for the density control of agents govern…
This work is devoted to the Lipschitz contraction and the long time behavior of certain Markov processes. These processes diffuse and jump. They can represent some natural phenomena like size of cell or data transmission over the Internet. Using a Feynman-Kac semigroup, we prove a bound in Wasserstein metric. This bound is explicit and optimal in the sense of Wasserstein curvature. This notion of curvature is relatively close to the notion of (coarse) Ricci curvature or spectral gap. Several consequences and examples are developed, including an $L^2$ spectral for general Markov processes, expl…
A deterministic walk in a random environment can be understood as a general random process with finite-range dependence that starts repeating a loop once it reaches a site it has visited before. Such process lacks the Markov property. We study the exponential decay of the probabilities that the walk will reach sites located far away from the origin. We also study a similar problem for the continuous analogue: the process that is a solution to an ODE with random coefficients. In this second model the environment also has "teleports" which are the regions from where the process can make disconti…
This paper presents a numerical method to calculate the value function for a general discounted impulse control problem for piecewise deterministic Markov processes. Our approach is based on a quantization technique for the underlying Markov chain defined by the post jump location and inter-arrival time. Convergence results are obtained and more importantly we are able to give a convergence rate of the algorithm. The paper is illustrated by a numerical example.
We consider random walks indexed by arbitrary finite random or deterministic trees. We derive a simple sufficient criterion which ensures that the maximal displacement of the tree-indexed random walk is determined by a single large jump. This criterion is given in terms of four quantities : the tail and the expectation of the random walk steps, the height of the tree and the number of its vertices. The results are applied to critical Galton--Watson trees with offspring distributions in the domain of attraction of a stable law.
We study a class of Piecewise Deterministic Markov Processes with state space Rd x E where E is a finite set. The continuous component evolves according to a smooth vector field that is switched at the jump times of the discrete coordinate. The jump rates may depend on the whole position of the process. Working under the general assumption that the process stays in a compact set, we detail a possible construction of the process and characterize its support, in terms of the solutions set of a differential inclusion. We establish results on the long time behaviour of the process, in relation to …
In this article we study a system of $N$ particles, each of them being defined by the couple of a position (in $\mathbb{R}^d$) and a so-called orientation which is an element of a compact Riemannian manifold. This orientation can be seen as a generalisation of the velocity in Vicsek-type models such as [Degond et al. 2017; Degond, Motsch 2008]. We will assume that the orientation of each particle follows a jump process whereas its position evolves deterministically between two jumps. The law of the jump depends on the position of the particle and the orientations of its neighbours. In the limi…
The main goal of this paper is to derive sufficient conditions for the existence of an optimal control strategy for the long run average continuous control problem of piecewise deterministic Markov processes (PDMP's) taking values in a general Borel space and with compact action space depending on the state variable. In order to do that we apply the so-called vanishing discount approach to obtain a solution to an average cost optimality inequality associated to the long run average cost problem. Our main assumptions are written in terms of some integro-differential inequalities related to the …
This paper considers adiabatic reduction in a model of stochastic gene expression with bursting transcription considered as a jump Markov process. In this model, the process of gene expression with auto-regulation is described by fast/slow dynamics. The production of mRNA is assumed to follow a compound Poisson process occurring at a rate depending on protein levels (the phenomena called bursting in molecular biology) and the production of protein is a linear function of mRNA numbers. When the dynamics of mRNA is assumed to be a fast process (due to faster mRNA degradation than that of protein…
This paper establishes limit theorems for a class of stochastic hybrid systems (continuous deterministic dynamic coupled with jump Markov processes) in the fluid limit (small jumps at high frequency), thus extending known results for jump Markov processes. We prove a functional law of large numbers with exponential convergence speed, derive a diffusion approximation and establish a functional central limit theorem. We apply these results to neuron models with stochastic ion channels, as the number of channels goes to infinity, estimating the convergence to the deterministic model. In terms of …
We propose a numerical method to approximate the value function for the optimal stopping problem of a piecewise deterministic Markov process (PDMP). Our approach is based on quantization of the post jump location---inter-arrival time Markov chain naturally embedded in the PDMP, and path-adapted time discretization grids. It allows us to derive bounds for the convergence rate of the algorithm and to provide a computable $ε$-optimal stopping time. The paper is illustrated by a numerical example.
We consider the multivariate point process determined by the crossing times of the components of a multivariate jump process through a multivariate boundary, assuming to reset each component to an initial value after its boundary crossing. We prove that this point process converges weakly to the point process determined by the crossing times of the limit process. This holds for both diffusion and deterministic limit processes. The almost sure convergence of the first passage times under the almost sure convergence of the processes is also proved. The particular case of a multivariate Stein pro…
We investigate piecewise deterministic Markov processes (PDMP), where the deterministic dynamics follows a scalar conservation law and random jumps in the system are characterized by changes in the flux function. We show under which assumptions we can guarantee the existence of a PDMP and conclude bounded variation estimates for sample paths. Finally, we apply this dynamics to a production and traffic model and use this framework to incorporate the well-known scattering of flux functions observed in data sets.
The Law of Large Numbers and CLT for Non-stationary Markov Jump Processes Exhibiting Time-of-Day Effects
本文为非平稳马尔可夫跳跃过程的累积奖励建立了大数定律和中心极限定理，涵盖连续及一次性奖励，并针对周期性和重置情形给出新结果，模拟验证了近似质量。
We consider the stochastic ranking process with space-time dependent unbounded jump rates for the particles. We prove that the joint empirical distribution of jump rate and scaled position converges almost surely to a deterministic distribution in the infinite particle limit. We assume topology of weak convergence for the space of distributions, which implies that the fluctuations among particles with different jump rates cancel in the limit. The results are proved by first finding an auxiliary stochastic ranking process, for which a strong law of large numbers is applied, and then applying a …
We provide quantitative bounds for the long time behavior of a class of Piecewise Deterministic Markov Processes with state space Rd \times E where E is a finite set. The continuous component evolves according to a smooth vector field that switches at the jump times of the discrete coordinate. The jump rates may depend on the whole position of the process. Under regularity assumptions on the jump rates and stability conditions for the vector fields we provide explicit exponential upper bounds for the convergence to equilibrium in terms of Wasserstein distances. As an example, we obtain converg…
We consider the stochastic ranking process with space-time dependent jump rates for the particles. The process is a simplified model of the time evolution of the rankings such as sales ranks at online bookstores. We prove that the joint empirical distribution of jump rate and scaled position converges almost surely to a deterministic distribution, and also the tagged particle processes converge almost surely, in the infinite particle limit. The limit distribution is characterized by a system of inviscid Burgers-like integral-partial differential equations with evaporation terms, and the limit …
We consider a class of stochastic dynamical systems, called piecewise deterministic Markov processes, with states $(x, \s)\in Ø\times \G$, $Ø$ being a region in $\bbR^d$ or the $d$--dimensional torus, $\G$ being a finite set. The continuous variable $x$ follows a piecewise deterministic dynamics, the discrete variable $\s$ evolves by a stochastic jump dynamics and the two resulting evolutions are fully--coupled. We study stationarity, reversibility and time--reversal symmetries of the process. Increasing the frequency of the $\s$--jumps, we show that the system behaves asymptotically as determ…
Exponential ergodicity of some Markov dynamical system with application to a Poisson driven stochastic differential equation
研究状态相关跳跃强度的分段确定性 Markov 过程的渐近性，证明其唯一不变分布在有界 Lipschitz 对偶距离中指数吸引，并推广了 Poisson 驱动随机微分方程跳跃过程的结果。
We give a short overview of recent results on a specific class of Markov process: the Piecewise Deterministic Markov Processes (PDMPs). We first recall the definition of these processes and give some general results. On more specific cases such as the TCP model or a model of switched vector fields, better results can be proved, especially as regards long time behaviour. We continue our review with an infinite dimensional example of neuronal activity. From the statistical point of view, these models provide specific challenges: we illustrate this point with the example of the estimation of the …
We study the existence of densities for distributions of piecewise deterministic Markov processes. We also obtain relationships between invariant densities of the continuous time process and that of the process observed at jump times. In our approach we use functional-analytic methods and the theory of linear operator semigroups. By imposing general conditions on the characteristics of a given Markov process, we show the existence of a substochastic semigroup describing the evolution of densities for the process and we identify its generator. Our main tool is a new perturbation theorem for sub…
The velocity jump Langevin process and its splitting scheme: long time convergence and numerical accuracy
本文对速度跳跃Langevin过程及其混合数值格式进行数值分析，证明了连续时间过程的指数遍历性、格式的2阶弱收敛阶，并给出了MCMC估计器的二次风险界。
Almost sure convergence of numerical approximations for Piecewise Deterministic Markov Processes
本文针对分段确定性马尔可夫过程，通过将随机跳跃时间转化为带随机阈值的常微分方程组首达时间问题，并利用确定性连续数值方法求解，证明了随机算法的几乎必然渐近收敛率与嵌入的确定性方法阶数相同。
扫码添加微信好友，提出您的宝贵建议 👇
💡 备注请填写：网站反馈
发表机构由 AI 从论文文本抽取，如发现错误，欢迎反馈修正建议。