We study a model of a polling system, that is, a collection of $d$ queues with a single server that switches from queue to queue. The service time distribution and arrival rates change randomly every time a queue is emptied. This model is mapped to a mathematically equivalent model of a random walk with random choice of transition probabilities, a model which is of independent interest. All our results are obtained using methods from the constructive theory of Markov chains. We determine conditions for the existence of polynomial moments of hitting times for the random walk. An unusual phenomenon of thickness of the region of null recurrence for both the random walk and the queueing model is also proved.
We consider a polling model with multiple stations, each with Poisson arrivals and a queue of infinite capacity. The service regime is exhaustive and there is Jacksonian feedback of served customers. What is new here is that when the server comes to a station it chooses the service rate and the feedback parameters at random; these remain valid during the whole stay of the server at that station. We give criteria for recurrence, transience and existence of the $s$th moment of the return time to the empty state for this model. This paper generalizes the model, when only two stations accept arriv…
A state-dependent 1-limited polling model with N queues is analyzed. The routing strategy generalizes the classical Markovian polling model, in the sense that two routing matrices are involved, the choice being made according to the state of the last visited queue. The stationary distribution of the position of the server is given. Ergodicity conditions are obtained by means of an associated dynamical system. Under rotational symmetry assumptions, average queue length and mean waiting times are computed.
In this paper, we deal with a two-queue polling system attended by a single server. The server visits the queues according to a Markovian routing mechnism. There are two-class customers in the first queue. Customers of each queue are served in the exhaustive discipline. For this model, we study the joint number of customers, the cycle time and the waiting times. we are also devoted to deriving the exact asymptotics for the scaled delay in the heavy-traffic scenario. In addition, the scaled delays with large switch-over times are discussed for the first time in the Markovian polling systems. Fi…
The paper analyzes the performance of tandem network of polling queue with setups. For a system with two-products and two-stations, we propose a new approach based on a partially-collapsible state-space characterization to reduce state-space complexity. In this approach, the size of the state-space is varied depending on the information needed to determine buffer levels and waiting times. We evaluate system performance under different system setting and comment on the numerical accuracy of the approach as well as provide managerial insights. Numerical results show that approach yields reliable…
This paper studies limit theorems for Markov Chains with general state space under conditions which imply subgeometric ergodicity. We obtain a central limit theorem and moderate deviation principles for additive not necessarily bounded functional of the Markov chains under drift and minorization conditions which are weaker than the Foster-Lyapunov conditions. The regeneration-split chain method and a precise control of the modulated moment of the hitting time to small sets are employed in the proof.
In this paper, we will develop a tool to analyze polling systems with the autonomous-server, the time-limited, and the k-limited service discipline. It is known that these disciplines do not satisfy the well-known branching property in polling system, therefore, hardly any exact result exists in the literature for them. Our strategy is to apply an iterative scheme that is based on relating in closed-form the joint queue-length at the beginning and the end of a server visit to a queue. These kernel relations are derived using the theory of absorbing Markov chains. Finally, we will show that our…
We study a class of Markov processes that combine local dynamics, arising from a fixed Markov process, with regenerations arising at a state-dependent rate. We give conditions under which such processes possess a given target distribution as their invariant measures, thus making them amenable for use within Monte Carlo methodologies. Since the regeneration mechanism can compensate the choice of local dynamics, while retaining the same invariant distribution, great flexibility can be achieved in selecting local dynamics, and the mathematical analysis is simplified. We give straightforward condi…
We prove two propositions with conditions that a system, which is described by a transient Markov chain, will display local stability. Examples of such systems include partly overloaded Jackson networks, partly overloaded polling systems, and overloaded multi-server queues with skill based service, under first come first served policy.
For a Harris ergodic Markov chain $(X_n)_{n\ge 0}$, on a general state space, started from the so called small measure or from the stationary distribution we provide optimal estimates for Orlicz norms of sums $\sum_{i=0}^τf(X_i)$, where $τ$ is the first regeneration time of the chain. The estimates are expressed in terms of other Orlicz norms of the function $f$ (wrt the stationary distribution) and the regeneration time $τ$ (wrt the small measure). We provide applications to tail estimates for additive functionals of the chain $(X_n)$ generated by unbounded functions as well as to classical l…
In 1987, J.W. Cohen analyzed the so-called Serve the Longest Queue (SLQ) queueing system, where a single server attends two non-symmetric $M/G/1$-type queues, exercising a non-preemptive priority switching policy. Cohen further analyzed in 1998 a non-symmetric 2-queue Markovian system, where newly arriving customers follow the Join the Shortest Queue (JSQ) discipline. The current paper generalizes and extends Cohen's works by studying a combined JSQ-SLQ model, and by broadening the scope of analysis to a non-symmetric 3-queue system, where arriving customers follow the JSQ strategy and a singl…
In this paper, we analyze a two-queue random time-limited Markov modulated polling model. In the first part of the paper, we investigate the fluid version: Fluid arrives at the two queues as two independent flows with deterministic rate. There is a single server that serves both queues at constant speeds. The server spends an exponentially distributed amount of time in each queue. After the completion of such a visit time to one queue, the server instantly switches to the other queue, i.e., there is no switchover time. For this model, we first derive the Laplace-Stieltjes Transform (LST) of th…
We consider a model, introduced by Boldrighini, Minlos and Pellegrinotti, of random walks in dynamical random environments on the integer lattice Z^d with d>=1. In this model, the environment changes over time in a Markovian manner, independently across sites, while the walker uses the environment at its current location in order to make the next transition. In contrast with the cluster expansions approach of Boldrighini, Minlos and Pellegrinotti, we follow a probabilistic argument based on regeneration times. We prove an annealed SLLN and invariance principle for any dimension, and provide a …
Consider a strong Markov process in continuous time, taking values in some Polish state space. Recently, Douc, Fort and Guillin (2009) introduced verifiable conditions in terms of a supermartingale property implying an explicit control of modulated moments of hitting times. We show how this control can be translated into a control of polynomial moments of abstract regeneration times which are obtained by using the regeneration method of Nummelin, extended to the time-continuous context. As a consequence, if a $p-$th moment of the regeneration times exists, we obtain non asymptotic deviation bo…
Let $X$ be a Harris recurrent strong Markov process in continuous time with general Polish state space $E,$ having invariant measure $μ.$ In this paper we use the regeneration method to derive non asymptotic deviation bounds for $$P_{x} (|\int_0^tf(X_s)ds|\geq t^{\frac12 + η} \ge)$$ in the positive recurrent case, for nice functions $f$ with $μ(f) =0 $ ($f$ must be a charge). We generalize these bounds to the fully null-recurrent case in the moderate deviations regime. We obtain a Gaussian contentration bound for all functions $f$ which are a charge. The rate of convergence is expressed in ter…
This paper surveys various results about Markov chains on general (non-countable) state spaces. It begins with an introduction to Markov chain Monte Carlo (MCMC) algorithms, which provide the motivation and context for the theory which follows. Then, sufficient conditions for geometric and uniform ergodicity are presented, along with quantitative bounds on the rate of convergence to stationarity. Many of these results are proved using direct coupling constructions based on minorisation and drift conditions. Necessary and sufficient conditions for Central Limit Theorems (CLTs) are also presente…
We introduce Path-ZVA: an efficient simulation technique for estimating the probability of reaching a rare goal state before a regeneration state in a (discrete-time) Markov chain. Standard Monte Carlo simulation techniques do not work well for rare events, so we use importance sampling; i.e., we change the probability measure governing the Markov chain such that transitions `towards' the goal state become more likely. To do this we need an idea of distance to the goal state, so some level of knowledge of the Markov chain is required. In this paper, we use graph analysis to obtain this knowled…
We consider the first exit time of a nonnegative Harris-recurrent Markov process from the interval $[0,A]$ as $A\to\infty$. We provide an alternative method of proof of asymptotic exponentiality of the first exit time (suitably standardized) that does not rely on embedding in a regeneration process. We show that under certain conditions the moment generating function of a suitably standardized version of the first exit time converges to that of $\Exp(1)$, and we connect between the standardizing constant and the quasi-stationary distribution (assuming it exists). The results are applied to the…
In the analysis of Markov chains and processes, it is sometimes convenient to replace an unbounded state space with a "truncated" bounded state space. When such a replacement is made, one often wants to know whether the equilibrium behavior of the truncated chain or process is close to that of the untruncated system. For example, such questions arise naturally when considering numerical methods for computing stationary distributions on unbounded state space. In this paper, we use the principle of "regeneration" to show that the stationary distributions of "fixed state" truncations converge in …
A rigorous and largely self-contained account of (a) the bread-and-butter concepts and techniques in Markov chain theory and (b) the long-term behaviour of chains. As much as possible, the treatment is probabilistic instead of analytical (I stay away from semigroup theory). Personally, I tend to find that the intuition lies with the former and not the latter. This manuscript is geared towards those interested in the use of Markov chains as models of real-life phenomena. For this reason, I focus on the type of chains most commonly encountered in practice (time-homogeneous, minimal, and right-co…
We consider random walks in dynamic random environments, with an environment generated by the time-reversal of a Markov process from the oriented percolation universality class. If the influence of the random medium on the walk is small in space-time regions where the medium is typical, we obtain a law of large numbers and an averaged central limit theorem for the walk via a regeneration construction under suitable coarse-graining. Such random walks occur naturally as spatial embeddings of ancestral lineages in spatial population models with local regulation. We verify that our assumptions hol…
