We consider perturbations of positive recurrent Markov modulated fluid models. In addition to the infinitesimal generator of the phases, we also perturb the rate matrix, and analyze the effect of those perturbations on the matrix of first return probabilities to the initial level. Our main contribution is the construction of a substitute for the matrix of first return probabilities, which enables us to analyze the effect of the perturbation under consideration.
We derive the Markov-modulated generalized Ornstein-Uhlenbeck process by embedding a Markov-modulated random recurrence equation in continuous time. The obtained process turns out to be the unique solution of a certain stochastic differential equation driven by a bivariate Markov-additive process. We present this stochastic differential equation as well as its solution explicitely in terms of the driving Markov-additive process. Moreover, we give necessary and sufficient conditions for strict stationarity of the Markov-modulated generalized Ornstein-Uhlenbeck process, and prove that its statio…
We consider a Markov-modulated Brownian motion $\{Y(t), ρ(t)\}$ with two boundaries at $0$ and $b > 0$, and allow for the controlling Markov chain $\{ρ(t)\}$ to instantaneously undergo a change of phase upon hitting either of the two boundaries at semi-regenerative epochs defined to be the first time the process reaches a boundary since it last hits the other boundary. We call this process a flexible Markov-modulated Brownian motion. Using the recently-established links between stochastic fluid models and Markov-modulated Brownian motions, we determine important characteristics of first exit t…
Ramaswami showed recently that standard Brownian motion arises as the limit of a family of Markov-modulated linear fluid processes. We pursue this analysis with a fluid approximation for Markov-modulated Brownian motion. Furthermore, we prove that the stationary distribution of a Markov-modulated Brownian motion reflected at zero is the limit from the well-analyzed stationary distribution of approximating linear fluid processes. Key matrices in the limiting stationary distribution are shown to be solutions of a new quadratic equation, and we describe how this equation can be efficiently solved…
We consider a simple regression model where a regressor is composed of order statistics and a noise is Markov-modulated. We introduce an empirical bridge of regression residuals and prove its weak convergence to a centered Gaussian process.
We extend to Markov-modulated Brownian motion (MMBM) the renewal approach which has been successfully applied to the analysis of Markov-modulated fluid models. It has recently been shown that MMBM may be expressed as the limit of a parameterized family of Markov-modulated fluid models. We prove that the weak convergence also holds for systems with two reflecting boundaries, one at zero and one at $b >0$, and that the stationary distributions of the approximating fluid models converge to the stationary distribution of the two-sided reflected MMBM. Thus, we obtain a new representation for the st…
In this paper, we study small noise asymptotics of Markov-modulated diffusion processes in the regime that the modulating Markov chain is rapidly switching. We prove the joint sample-path large deviations principle for the Markov-modulated diffusion process and the occupation measure of the Markov chain (which evidently also yields the large deviations principle for each of them separately by applying the contraction principle). The structure of the proof is such that we first prove exponential tightness, and then establish a local large deviations principle (where the latter part is split int…
In this paper we study a reflected Markov-modulated Brownian motion with a two sided reflection in which the drift, diffusion coefficient and the two boundaries are (jointly) modulated by a finite state space irreducible continuous time Markov chain. The goal is to compute the stationary distribution of this Markov process, which in addition to the complication of having a stochastic boundary can also include jumps at state change epochs of the underlying Markov chain because of the boundary changes. We give the general theory and then specialize to the case where the underlying Markov chain h…
Generalization of the Lorden's inequality is an excellent tool for obtaining strong upper bounds for the convergence rate for various complicated stochastic models. This paper demonstrates a method for obtaining such bounds for some generalization of the Markov modulated Poisson process (MMPP). The proposed method can be applied to reliability and queuing theory.
We analyze the stationary distribution of regulated Markov modulated Brownian motions (MMBM) modified so that their evolution is slowed down when the process reaches level zero --- level zero is said to be {\em sticky}. To determine the stationary distribution, we extend to MMBMs a construction of Brownian motion with sticky boundary, and we follow a Markov-regenerative approach similar to the one developed in past years in the context of quasi-birth-and-death processes and fluid queues. We also rely on recent work showing that Markov-modulated Brownian motions may be analyzed as limits of a p…
We construct a stochastic fluid process with an underlying piecewise deterministic Markov process (PDMP) akin to the one used in the construction of the rational arrival process (RAP), which we call the RAP-modulated fluid process. As opposed to the classic stochastic fluid process driven by a Markov jump process, the underlying PDMP of a RAP-modulated fluid process has a continuous state space and is driven by matrix parameters which may not be related to an intensity matrix. Through novel techniques we show how well-known formulae associated to the classic stochastic fluid process, such as f…
In this paper we study limit behavior for a Markov-modulated (MM) binomial counting process, also called a binomial counting process under regime switching. Such a process naturally appears in the context of credit risk when multiple obligors are present. Markov-modulation takes place when the failure/default rate of each individual obligor depends on an underlying Markov chain. The limit behavior under consideration occurs when the number of obligors increases unboundedly, and/or by accelerating the modulating Markov process, called rapid switching. We establish diffusion approximations, obta…
In Latouche and Nguyen (2015), the authors constructed a sequence of stochastic fluid processes and showed that it converges weakly to a Markov-modulated Brownian motion (MMBM). Here, we construct a different sequence of stochastic fluid processes and show that it converges strongly to an MMBM. To the best of our knowledge, this is the first result on strong convergence to a Markov-modulated Brownian motion. We also prove that the rate of this almost sure convergence is $o(n^{-1/2} \log n)$. When reduced to the special case of standard Brownian motion, our convergence rate is an improvement ov…
We consider a Markov-modulated Brownian motion reflected to stay in a strip [0,B]. The stationary distribution of this process is known to have a simple form under some assumptions. We provide a short probabilistic argument leading to this result and explaining its simplicity. Moreover, this argument allows for generalizations including the distribution of the reflected process at an independent exponentially distributed epoch. Our second contribution concerns transient behavior of the reflected system. We identify the joint law of the processes t,X(t),J(t) at inverse local times.
We consider a Markov modulated fluid network with a finite number of stations. We are interested in the tail asymptotics behavior of the stationary distribution of its buffer content process. Using two different approaches, we derive upper and lower bounds for the stationary tail decay rate in various directions. Both approaches are based on a well-known time-evolution formula of a Markov process, so-called Dynkin's formula, where a key ingredient is a suitable choice of test functions. Those results show how multidimensional tail asymptotics can be studied for the more than two-dimensional ca…
In this work, we focus on the stationary analysis of a specific class of continuous time Markov-modulated reflected random walks in the quarter plane with applications in the modelling of two-node Markov-modulated queueing networks with coupled queues. The transition rates of the two-dimensional process depend on the state of a finite state Markovian background process. Such a modulation is space homogeneous in the set of inner states of the two-dimensional lattice but may be different in the set of states at its boundaries. To obtain the stationary distribution, we apply the power series appr…
The production of molecules in a chemical reaction network is modelled as a Poisson process with a Markov-modulated arrival rate and an exponential decay rate. We analyze the distributional properties of $M$, the number of molecules, under specific time-scaling; the background process is sped up by $N^α$, the arrival rates are scaled by $N$, for $N$ large. A functional central limit theorem is derived for $M$, which after centering and scaling, converges to an Ornstein-Uhlenbeck process. A dichotomy depending on $α$ is observed. For $α\leq1$ the parameters of the limiting process contain the d…
We consider a discrete-time Markov chain $(X^t,Y^t)$, $t=0,1,2,...$, where the $X$-component forms a Markov chain itself. Assume that $(X^t)$ is Harris-ergodic and consider an auxiliary Markov chain ${\hat{Y}^t}$ whose transition probabilities are the averages of transition probabilities of the $Y$-component of the $(X,Y)$-chain, where the averaging is weighted by the stationary distribution of the $X$-component. We first provide natural conditions in terms of test functions ensuring that the $\hat{Y}$-chain is positive recurrent and then prove that these conditions are also sufficient for pos…
The objective of this paper is to give conditions ensuring that the backward partial integro differential equation associated with a multidimensional jump-diffusion with a pure jump component has a unique classical solution; that is the solution is continuous, twice differentiable in the diffusion component and differentiable in time. Our proof uses a probabilistic arguments and extends the results of Pham (1998) to processes with a pure jump component where the jump intensity is modulated by a diffusion process. This result is particularly useful in some applications to pricing and hedging of…
This paper studies the diffusion limit for a network of infinite-server queues operating under Markov modulation (meaning that the system's parameters depend on an autonomously evolving background process). In previous papers on (primarily single-node) queues with Markov modulation, two variants were distinguished: one in which the server speed is modulated, and one in which the service requirement is modulated (i.e., depends on the state of the background process upon arrival). The setup of the present paper, however, is more general, as we allow both the server speed and the service requirem…
This paper studies a scheduling control problem for a single-server multiclass queueing network in heavy traffic, operating in a changing environment. The changing environment is modeled as a finite state Markov process that modulates the arrival and service rates in the system. Various cases are considered: fast changing environment, fixed environment and slow changing environment. In each of the cases, using weak convergence analysis, in particular functional limit theorems for renewal processes and ergodic Markov processes, it is shown that an appropriate "averaged" version of the classical…
In this paper we present elementary computations for some Markov modulated counting processes, also called counting processes with regime switching. Regime switching has become an increasingly popular concept in many branches of science. In finance, for instance, one could identify the background process with the `state of the economy', to which asset prices react, or as an identification of the varying default rate of an obligor. The key feature of the counting processes in this paper is that their intensity processes are functions of a finite state Markov chain. This kind of processes can be…
Let $\{\boldsymbol{X}_n\}$ be a discrete-time $d$-dimensional process on $\mathbb{Z}_+^d$ with a supplemental (background) process $\{J_n\}$ on a finite set and assume the joint process $\{\boldsymbol{Y}_n\}=\{(\boldsymbol{X}_n,J_n)\}$ to be Markovian. Then, the process $\{\boldsymbol{X}_n\}$ can be regarded as a kind of reflecting random walk (RRW for short) in which the transition probabilities of the RRW are modulated according to the state of the background process $\{J_n\}$; we assume this modulation is space-homogeneous inside $\mathbb{Z}_+^d$ and on each boundary face of $\mathbb{Z}_+^d…
In this paper we study the Markov-modulated M/M/$\infty$ queue, with a focus on the correlation structure of the number of jobs in the system. The main results describe the system's asymptotic behavior under a particular scaling of the model parameters in terms of a functional central limit theorem. More specifically, relying on the martingale central limit theorem, this result is established, covering the situation in which the arrival rates are sped up by a factor $N$ and the transition rates of the background process by $N^α$, for some $α>0$. The results reveal an interesting dichotomy, wit…
We study optimal investment strategies that maximize expected utility from consumption and terminal wealth in a pure-jump asset price model with Markov-modulated (regime switching) jump-size distributions. We give sufficient conditions for existence of optimal policies and find closed-form expressions for the optimal value function for agents with logarithmic and fractional power (CRRA) utility in the case of two-state Markov chains. The main tools are convex duality techniques, stochastic calculus for pure-jump processes and explicit formulae for the moments of telegraph processes with Markov…
Many networking-related settings can be modeled by Markov-modulated infinite-server systems. In such models, the customers' arrival rates and service rates are modulated by a Markovian background process, additionally, there are infinitely many servers (and consequently the resulting model is often used as a proxy for the corresponding many-server model). The Markov-modulated infinite-server model hardly allows any explicit analysis, apart from results in terms of systems of (ordinary or partial) differential equations for the underlying probability generating functions, and recursions to obta…
We analyze asymptotically a differential-difference equation, that arises in a Markov-modulated fluid model. We use singular perturbation methods to analyze the problem with appropriate scalings of the two state variables. In particular, the ray method and asymptotic matching are used.
In this paper we propose a semi-Markov modulated model of interest rates. We assume that the switching process is a semi-Markov process with finite state space E and the modulated process is a diffusive process. We derive recursive equations for the higher order moments of the discount factor and we describe a Monte Carlo al- gorithm to execute simulations. The results are specialized to classical models as those by Vasicek, Hull and White and CIR with a semi-Markov modulation.
Markov modulation is versatile in generalization for making a simple stochastic model which is often analytically tractable to be more flexible in application. In this spirit, we modulate a two dimensional reflecting skip-free random walk in such a way that its state transitions in the boundary faces and interior of a nonnegative integer quadrant are controlled by Markov chains. This Markov modulated model is referred to as a 2d-QBD process according to Ozaw (2013). We are interested in the tail asymptotics of its stationary distribution, which has been well studied when there is no Markov mod…
This paper deals with a single-server queue with modulated arrivals, service requirements and service capacity. In our first result, we derive the mean of the total workload assuming generally distributed service requirements and any service discipline which does not depend on the modulating environment. We then show that the workload is exponentially distributed under heavy-traffic scaling. In our second result, we focus on the discriminatory processor sharing (DPS) discipline. Assuming exponential, class-dependent service requirements, we show that the joint distribution of the queue lengths…
This paper focuses on an infinite-server queue modulated by an independently evolving finite-state Markovian background process, with transition rate matrix $Q\equiv(q_{ij})_{i,j=1}^d$. Both arrival rates and service rates are depending on the state of the background process. The main contribution concerns the derivation of central limit theorems for the number of customers in the system at time $t\ge 0$, in the asymptotic regime in which the arrival rates $λ_i$ are scaled by a factor $N$, and the transition rates $q_{ij}$ by a factor $N^α$, with $α\in \mathbb R^+$. The specific value of $α$ h…
We study multiclass many-server queues for which the arrival, service and abandonment rates are all modulated by a common finite-state Markov process. We assume that the system operates in the "averaged" Halfin-Whitt regime, which means that it is critically loaded in the average sense, although not necessarily in each state of the Markov process. We show that under any static priority policy, the Markov-modulated diffusion-scaled queueing process is geometrically ergodic. This is accomplished by employing a solution to an associated Poisson equation in order to construct a suitable Lyapunov f…
We study Markov-modulated affine processes (abbreviated MMAPs), a class of Markov processes that are created from affine processes by allowing some of their coefficients to be a function of an exogenous Markov process. MMAPs allow for richer models in various applications. At the same time MMAPs largely preserve the tractability of standard affine processes, as their characteristic function has a computationally convenient functional form. Our setup is a substantial generalization of earlier work, since we consider the case where the generator of the exogenous process $X$ is an unbounded opera…
This paper examines a discrete-time queuing system with applications to telecommunications traffic. The arrival process is a particular Markov modulated process which belongs to the class of discrete batched Markovian arrival processes. The server process is a single server deterministic queue. A closed form exact solution is given for the expected queue length and delay. A simple system of equations is given for the probability of the queue exceeding a given length.
We analyze asymptotically a differential-difference equation, that arises in a Markov-modulated fluid model. Here there are N identical sources that turn "on" and "off", and when "on" they generate fluid at unit rate into a buffer, which process the fluid at a rate c < N. In the steady state limit, the joint probability distribution of the buffer content and the number of active sources satisfies a system of N + 1 ODEs, that can also be viewed as a differential-difference equation analogous to a backward/forward parabolic PDE. We use singular perturbation methods to analyze the problem for N l…
Stochastic models for performance analysis, optimization and control of queues hinge on a multitude of alternatives for input point processes. In case of bursty traffic, one very popular model is the \textit{Markov Modulated Poisson Process} (MMPP), however it is not the only option. Here, we introduce an alternative that we call \textit{Markovian transition counting process} (MTCP). The latter is a point process counting the number of transitions of a finite continuous-time Markov chain. For a given MTCP one can establish an MMPP with the same first and second moments of counts. In this paper…
We consider a self-exciting counting process, the parameters of which depend on a hidden finite-state Markov chain. We derive the optimal filter and smoother for the hidden chain based on observation of the jump process. This filter is in closed form and is finite dimensional. We demonstrate the performance of this filter both with simulated data, and by analysing the `flash crash' of 6th May 2010 in this framework.
We study ergodic properties of a class of Markov-modulated general birth-death processes under fast regime switching. The first set of results concerns the ergodic properties of the properly scaled joint Markov process with a parameter that is taken large. Under very weak hypotheses, we show that if the averaged process is exponentially ergodic for large values of the parameter, then the same applies to the original joint Markov process. The second set of results concerns steady-state diffusion approximations, under the assumption that the 'averaged' fluid limit exists. Here, we establish conv…
The paper proposes a class of financial market models which are based on inhomogeneous telegraph processes and jump diffusions with alternating volatilities. It is assumed that the jumps occur when the tendencies and volatilities are switching. We argue that such a model captures well the stock price dynamics under periodic financial cycles. The distribution of this process is described in detail. For this model we obtain the structure of the set of martingale measures. This incomplete model can be completed by adding another asset based on the same sources of randomness. Explicit closed-form …
We consider a discrete-time two-dimensional process $\{(X_{1,n},X_{2,n})\}$ on $\mathbb{Z}^2$ with a background process $\{J_n\}$ on a finite set $S_0$, where individual processes $\{X_{1,n}\}$ and $\{X_{2,n}\}$ are both skip free. We assume that the joint process $\{\boldsymbol{Y}_n\}=\{(X_{1,n},X_{2,n},J_n)\}$ is Markovian and that the transition probabilities of the two-dimensional process $\{(X_{1,n},X_{2,n})\}$ vary according to the state of the background process $\{J_n\}$. This modulation is assumed to be space homogeneous. We refer to this process as a two-dimensional skip-free Markov …
We consider a discrete-time $d$-dimensional process $\{\boldsymbol{X}_n\}=\{(X_{1,n},X_{2,n},...,X_{d,n})\}$ on $\mathbb{Z}^d$ with a background process $\{J_n\}$ on a countable set $S_0$, where individual processes $\{X_{i,n}\},i\in\{1,2,...,d\},$ are skip free. We assume that the joint process $\{\boldsymbol{Y}_n\}=\{(\boldsymbol{X}_n,J_n)\}$ is Markovian and that the transition probabilities of the $d$-dimensional process $\{\boldsymbol{X}_n\}$ vary according to the state of the background process $\{J_n\}$. This modulation is assumed to be space homogeneous. We refer to this process as a $…
The conditional intensity (CI) of a counting process $Y_t$ is based on the minimal knowledge $\mathcal{F}_t^Y$, i.e., on the observation of $Y_t$ alone. Prominently, the mutual information rate of a signal and its Poisson channel output is a difference functional between the CI and the intensity that has full knowledge about the input. While the CI of Markov-modulated Poisson processes evolves according to Snyder's filter, self-exciting processes, e.g., Hawkes processes, specify the CI via the history of $Y_t$. The emergence of the CI as a self-contained stochastic process prompts us to bring …
A popular method to compute first-passage probabilities in continuous-time Markov chains is by numerically inverting their Laplace transforms. Past decades, the scientific computing community has developed excellent numerical methods for solving problems governed by partial differential equations (PDEs), making the availability of a Laplace transform not necessary here for computational purposes. In this study we demonstrate that numerical PDE problem solvers are suitable for computing first passage times, and can be very efficient for this purpose. By doing extensive computational experiments…
We consider a linear stochastic fluid network under Markov modulation, with a focus on the probability that the joint storage level attains a value in a rare set at a given point in time. The main objective is to develop efficient importance sampling algorithms with provable performance guarantees. For linear stochastic fluid networks without modulation, we prove that the number of runs needed (so as to obtain an estimate with a given precision) increases polynomially (whereas the probability under consideration decays essentially exponentially); for networks operating in the slow modulation r…
Let $X$ be the constrained random walk on ${\mathbb Z}_+^2$ having increments $(1,0)$, $(-1,1)$, $(0,-1)$ with jump probabilities $λ(M_k)$, $μ_1(M_k)$, and $μ_2(M_k)$ where $M$ is an irreducible aperiodic finite state Markov chain. The process $X$ represents the lengths of two tandem queues with arrival rate $λ(M_k)$, and service rates $μ_1(M_k)$, and $μ_2(M_k)$. We assume that the average arrival rate with respect to the stationary measure of $M$ is less than the average service rates, i.e., $X$ is assumed stable. Let $τ_n$ be the first time when the sum of the components of $X$ equals $n$ fo…
Fluid limit techniques have become a central tool to analyze queueing networks over the last decade, with applications to performance analysis, simulation and optimization. In this paper, some of these techniques are extended to a general class of skip-free Markov chains. As in the case of queueing models, a fluid approximation is obtained by scaling time, space and the initial condition by a large constant. The resulting fluid limit is the solution of an ordinary differential equation (ODE) in ``most'' of the state space. Stability and finer ergodic properties for the stochastic model then fo…
Markov-modulated Lévy processes with two different regimes of restarting are studied. These regimes correspond to the completely renewed process and to the process of Markov modulation, accompanied by jumps. We give explicit expressions for the Lévy-Khintchine exponent in the case of a two-state underlying Markov chain. For the renewal case, the limit distributions (as $t\to\infty$) are obtained. In the case of processes with jumps, we present some results for the exponential functional.
A rescaled Markov chain converges uniformly in probability to the solution of an ordinary differential equation, under carefully specified assumptions. The presentation is much simpler than those in the outside literature. The result may be used to build parsimonious models of large random or pseudo-random systems.
Introduced by Bean and O'Reilly (2014), a stochastic fluid-fluid process is a Markov processes $\{X_t, Y_t, φ_t\}_{t \geq 0}$, where the first fluid $X_t$ is driven by the Markov chain $φ_t$, and the second fluid $Y_t$ is driven by $φ_t$ as well as by $X_t$. That paper derived a closed-form expression for the joint stationary distribution, given in terms of operators acting on measures, which does not lend itself easily to numerical computations. Here, we construct a discontinuous Galerkin method for approximating this stationary distribution, and illustrate the methodology using an on-off ban…
