We consider a change-point detection problem for a simple class of Piecewise Deterministic Markov Processes (PDMPs). A continuous-time PDMP is observed in discrete time and through noise, and the aim is to propose a numerical method to accurately detect both the date of the change of dynamics and the new regime after the change. To do so, we state the problem as an optimal stopping problem for a partially observed discrete-time Markov decision process taking values in a continuous state space and provide a discretization of the state space based on quantization to approximate the value function and build a tractable stopping policy. We provide error bounds for the approximation of the value function and numerical simulations to assess the performance of our candidate policy.
Optimal control of infinite-dimensional Piecewise Deterministic Markov Processes: a BSDE approach. Application to the control of an excitable cell membrane
A Piecewise Deterministic Markov Toy Model for Traffic/Maintenance and Associated Hamilton-Jacobi Integrodifferential Systems on Networks
我们研究了无限时域中的最优控制问题，当动态属于一类特定的分段确定性马尔可夫过程约束的星形网络（受交通模型的启发）。我们采用[H. M. Soner具有状态空间约束的最优控制。二. SIAM J. Control Optim.，24（6）：1110.1122，1986]证明了值函数的正则性和动态规划原理。扩展网络和Krylov的“摇动系数”方法，我们证明了值函数可以被看作是一个线性化的最优化问题集的解决方案上的一组方便的概率测度。该方法完全依赖于粘度参数。作为一个副产品，对偶制定保证值函数是点态上确界相关的Hamilton-Jacobi积分微分系统的正则子解。这确保值函数满足Perron对黏性解的（唯一）候选的preconization。最后，我们证明了同样的线性化可以通过结合经典（无约束）问题的线性化和成本惩罚。后一种方法适用于非常一般的近可行系统（可能没有进一步的可控性）和不连续成本。
Absorbing Markov Decision Processes: Geometric Properties and Sufficiency of Finite Mixtures of Deterministic Policies
Bordeaux INP（波尔多国立高等电子电气工程师学院；波尔多大学Inria中心，ASTRAL团队；IMB，波尔多数学研究所，波尔多大学）；Inria centre at the University of Bordeaux, Team: ASTRAL（统计学系，UNED（西班牙国立远程教育大学），马德里）；IMB, Institut de Mathématiques de Bordeaux, Université de Bordeaux；Statistics Department, UNED, Madrid
