v1: 28 pages, 8 figures; v2: 31 pages, 9 figures, major revision after review: added, modified, and removed material (no results dropped, results added), moved proofs to an appendix
When the initial and transition probabilities of a finite Markov chain in discrete time are not well known, we should perform a sensitivity analysis. This can be done by considering as basic uncertainty models the so-called credal sets that these probabilities are known or believed to belong to, and by allowing the probabilities to vary over such sets. This leads to the definition of an imprecise Markov chain. We show that the time evolution of such a system can be studied very efficiently using so-called lower and upper expectations, which are equivalent mathematical representations of credal sets. We also study how the inferred credal set about the state at time n evolves as n goes to infinity: under quite unrestrictive conditions, it converges to a uniquely invariant credal set, regardless of the credal set given for the initial state. This leads to a non-trivial generalisation of the classical Perron-Frobenius Theorem to imprecise Markov chains.
We consider the problem of characterising expected hitting times and hitting probabilities for imprecise Markov chains. To this end, we consider three distinct ways in which imprecise Markov chains have been defined in the literature: as sets of homogeneous Markov chains, as sets of more general stochastic processes, and as game-theoretic probability models. Our first contribution is that all these different types of imprecise Markov chains have the same lower and upper expected hitting times, and similarly the hitting probabilities are the same for these three types. Moreover, we provide a ch…
We study the problem of characterizing the expected hitting times for a robust generalization of continuous-time Markov chains. This generalization is based on the theory of imprecise probabilities, and the models with which we work essentially constitute sets of stochastic processes. Their inferences are tight lower- and upper bounds with respect to variation within these sets. We consider three distinct types of these models, corresponding to different levels of generality and structural independence assumptions on the constituent processes. Our main results are twofold; first, we demonstrat…
We present an algorithm that can efficiently compute a broad class of inferences for discrete-time imprecise Markov chains, a generalised type of Markov chains that allows one to take into account partially specified probabilities and other types of model uncertainty. The class of inferences that we consider contains, as special cases, tight lower and upper bounds on expected hitting times, on hitting probabilities and on expectations of functions that are a sum or product of simpler ones. Our algorithm exploits the specific structure that is inherent in all these inferences: they admit a gene…
In recent years probabilistic model checking has become an important area of research because of the diffusion of computational systems of stochastic nature. Despite its great success, standard probabilistic model checking suffers the limitation of requiring a sharp specification of the probabilities governing the model behaviour. The theory of imprecise probabilities offers a natural approach to overcome such limitation by a sensitivity analysis with respect to the values of these parameters. However, only extensions based on discrete-time imprecise Markov chains have been considered so far f…
不精确连续时间马尔可夫链是一种鲁棒的连续时间马尔可夫链，允许部分指定的时间依赖的参数。计算它们的推论需要求解非线性微分方程。由于该解没有通用的解析表达式，因此有效的数值逼近方法对该模型的适用性至关重要。本文从两个方面改进了Krak et al.（2016）的一致逼近方法，并提出了一种新的更有效的自适应逼近方法。对于遍历链，我们还提供了一种方法，使我们能够近似平稳分布到任何所需的最大误差。
We study the limit behaviour of upper and lower bounds on expected time averages in imprecise Markov chains; a generalised type of Markov chain where the local dynamics, traditionally characterised by transition probabilities, are now represented by sets of 'plausible' transition probabilities. Our main result is a set of necessary and sufficient conditions under which these upper and lower bounds, called upper and lower expected time averages, will converge as time progresses towards infinity to limit values that do not depend on the process' initial state. Remarkably, our conditions are cons…
We present a novel algorithm to solve a non-linear system of equations, whose solution can be interpreted as a tight lower bound on the vector of expected hitting times of a Markov chain whose transition probabilities are only partially specified. We also briefly sketch how this method can be modified to solve a conjugate system of equations that gives rise to the corresponding upper bound. We prove the correctness of our method, and show that it converges to the correct solution in a finite number of steps under mild conditions on the system. We compare the runtime complexity of our method to…
The theory of imprecise Markov chains has achieved significant progress in recent years. Its applicability, however, is still very much limited, due in large part to the lack of efficient computational methods for calculating higher-dimensional models. The high computational complexity shows itself especially in the calculation of the imprecise version of the Kolmogorov backward equation. The equation is represented at every point of an interval in the form of a minimization problem, solvable merely with linear programming techniques. Consequently, finding an exact solution on an entire interv…
Imprecise Transition Matrices for Markov Cohort Models: Lower and Upper Expectations with a Practical Health Economic Application
Medtronic Trading Sàrl（美敦力贸易公司）；Center for Evidence Synthesis in Health, Department of Health Services, Policy, and Practice, Brown University（布朗大学健康服务、政策与实践系健康证据综合中心）
We study the limit behaviour of upper and lower bounds on expected time averages in imprecise Markov chains; a generalised type of Markov chain where the local dynamics, traditionally characterised by transition probabilities, are now represented by sets of `plausible' transition probabilities. Our first main result is a necessary and sufficient condition under which these upper and lower bounds, called upper and lower expected time averages, will converge as time progresses towards infinity to limit values that do not depend on the process' initial state. Our condition is considerably weaker …
We investigate how basic probability inequalities can be extended to an imprecise framework, where (precise) probabilities and expectations are replaced by imprecise probabilities and lower/upper previsions. We focus on inequalities giving information on a single bounded random variable $X$, considering either convex/concave functions of $X$ (Jensen's inequalities) or one-sided bounds such as $(X\geq c)$ or $(X\leq c)$ (Markov's and Cantelli's inequalities). As for the consistency of the relevant imprecise uncertainty measures, our analysis considers coherence as well as weaker requirements, n…
We use the functional It{ô} calculus to prove that the solution of a BSDE with singular terminal condition is continuous at the terminal time. Hence we extend known results for a non-Markovian terminal condition.
Nonlinear Markov chains with finite state space have been introduced in Kolokoltsov (2010). The characteristic property of these processes is that the transition probabilities do not only depend on the state, but also on the distribution of the process. In this paper we provide first results regarding their invariant distributions and long-term behaviour. We will show that under a continuity assumption an invariant distribution exists. Moreover, we provide a sufficient criterion for the uniqueness of the invariant distribution that relies on the Brouwer degree. Thereafter, we will present exam…
This paper demonstrates a lower and upper solution method to investigate the asymptotic behaviour of the conservative reaction-diffusion systems associated with Markovian process algebra models. In particular, we have proved the uniform convergence of the solution to its constant equilibrium for a case study as time tends to infinity, together with experimental results illustrations.
We present formulas for quasi-ergodic limits of finite absorbing Markov chains. Since the irreducible case has been solved in 1965 by Darroch and Seneta, we focus on the reducible case, and our results are based on a very precise asymptotic analysis of the (exponential and polynomial) growth behaviour along admissible paths
We consider the discrete-time migration-recombination equation, a deterministic, nonlinear dynamical system that describes the evolution of the genetic type distribution of a population evolving under migration and recombination in a law of large numbers setting. We relate this dynamics (forward in time) to a Markov chain, namely a labelled partitioning process, backward in time. This way, we obtain a stochastic representation of the solution of the migration-recombination equation. As a consequence, one obtains an explicit solution of the nonlinear dynamics, simply in terms of powers of the t…
The goal of the paper is to describe the large time behaviour of a Markov process associated with a symmetric diffusion in a high-contrast random environment and to characterize the limit semigroup and the limit process under the diffusive scaling.
Motivated by the study of a parasite infection in a cell line, we introduce a general class of Markov processes for the modelling of population dynamics. The population process evolves as a diffusion with positive jumps whose rate is a function of the population size. It also undergoes catastrophic events which kill a fraction of the population, at a rate depending on the population state. We study the long time behaviour of this class of processes.
