In this paper we consider a control problem for a Partially Observable Piecewise Deterministic Markov Process of the following type: After the jump of the process the controller receives a noisy signal about the state and the aim is to control the process continuously in time in such a way that the expected discounted cost of the system is minimized. We solve this optimization problem by reducing it to a discrete-time Markov Decision Process. This includes the derivation of a filter for the unobservable state. Imposing sufficient continuity and compactness assumptions we are able to prove the existence of optimal policies and show that the value function satisfies a fixed point equation. A generic application is given to illustrate the results.
This paper presents with justifications a technique that is useful for the study of piecewise deterministic Markov decision processes (PDMDPs) with general policies and unbounded transition intensities. This technique produces an auxiliary PDMDP from the original one. As to be discussed and claified, the auxiliary PDMDP possesses certain desired properties, which may not be possessed by the original PDMDP. Moreover, the performance measure of any policy in the original PDMDP can be replicated by the auxiliary PDMDP for a large class of performance criteria. As an application, we apply this tec…
Optimal control of infinite-dimensional Piecewise Deterministic Markov Processes: a BSDE approach. Application to the control of an excitable cell membrane
In piecewise-deterministic Markov processes (PDMPs) the state of a finite-dimensional system evolves continuously, but the evolutive equation may change randomly as a result of discrete switches. A running cost is integrated along the corresponding piecewise-deterministic trajectory up to the termination to produce the cumulative cost of the process. We address three natural questions related to uncertainty in cumulative cost of PDMP models: (1) how to compute the Cumulative Distribution Function (CDF) of the cumulative cost when the switching rates are fully known; (2) how to accurately bound…
A Piecewise Deterministic Markov Toy Model for Traffic/Maintenance and Associated Hamilton-Jacobi Integrodifferential Systems on Networks
我们研究了无限时域中的最优控制问题，当动态属于一类特定的分段确定性马尔可夫过程约束的星形网络（受交通模型的启发）。我们采用[H. M. Soner具有状态空间约束的最优控制。二. SIAM J. Control Optim.，24（6）：1110.1122，1986]证明了值函数的正则性和动态规划原理。扩展网络和Krylov的“摇动系数”方法，我们证明了值函数可以被看作是一个线性化的最优化问题集的解决方案上的一组方便的概率测度。该方法完全依赖于粘度参数。作为一个副产品，对偶制定保证值函数是点态上确界相关的Hamilton-Jacobi积分微分系统的正则子解。这确保值函数满足Perron对黏性解的（唯一）候选的preconization。最后，我们证明了同样的线性化可以通过结合经典（无约束）问题的线性化和成本惩罚。后一种方法适用于非常一般的近可行系统（可能没有进一步的可控性）和不连续成本。
In this manuscript, we investigate symbolic abstractions that capture the behavior of piecewise-affine systems under input constraints and bounded external noise. This is accomplished by considering local affine feedback controllers that are jointly designed with the symbolic model, which ensures that an alternating simulation relation between the system and the abstraction holds. The resulting symbolic system is called a state-feedback abstraction and we show that it can be deterministic even when the original piecewise-affine system is unstable and non-deterministic. One benefit of this appr…
Consider a non-autonomous continuous-time linear system in which the time-dependent matrix determining the dynamics is piecewise constant and takes finitely many values $A_1, \dotsc, A_N$. This paper studies the equality cases between the maximal Lyapunov exponent associated with the set of matrices $\{A_1, \dotsc, A_N\}$, on the one hand, and the corresponding ones for piecewise deterministic Markov processes with modes $A_1, \dotsc, A_N$, on the other hand. A fundamental step in this study consists in establishing a result of independent interest, namely, that any sequence of Markov processe…
