The paper studies closed queueing networks containing a server station and $k$ client stations. The server station is an infinite server queueing system, and client stations are single-server queueing systems with autonomous service, i.e. every client station serves customers (units) only at random instants generated by a strictly stationary and ergodic sequence of random variables. The total number of units in the network is $N$. The expected times between departures in client stations are $(Nμ_j)^{-1}$. After a service completion in the server station, a unit is transmitted to the $j$th client station with probability $p_{j}$ $(j=1,2,...,k)$, and being processed in the $j$th client station, the unit returns to the server station. The network is assumed to be in a semi-Markov environment. A semi-Markov environment is defined by a finite or countable infinite Markov chain and by sequences of independent and identically distributed random variables. Then the routing probabilities $p_{j}$ $(j=1,2,...,k)$ and transmission rates (which are expressed via parameters of the network) depend on a Markov state of the environment. The paper studies the queue-length processes in client stations of this network and is aimed to the analysis of performance measures associated with this network. The questions risen in this paper have immediate relation to quality control of complex telecommunication networks, and the obtained results are expected to lead to the solutions to many practical problems of this area of research.
In this note, a general approach to the study of non-stationary Markov chains with catastrophes and the corresponding queuing models is considered, as well as to obtain estimates of the limiting regime itself. As an illustration, an example of a queuing model is studied.
In this paper, we present a numerical framework for constructing bounds on stationary performance measures of random walks in the positive orthant using the Markov reward approach. These bounds are established in terms of stationary performance measures of a perturbed random walk whose stationary distribution is known explicitly. We consider random walks in an arbitrary number of dimensions and with a transition probability structure that is defined on an arbitrary partition of the positive orthant. Within each component of this partition the transition probabilities are homogeneous. This enab…
Queueing networks are notoriously difficult to analyze sans both Markovian and stationarity assumptions. Much of the theoretical contribution towards performance analysis of time-inhomogeneous single class queueing networks has focused on Markovian networks, with the recent exception of work in Liu and Whitt (2011) and Mandelbaum and Ramanan (2010). In this paper, we introduce transitory queueing networks as a model of inhomogeneous queueing networks, where a large, but finite, number of jobs arrive at queues in the network over a fixed time horizon. The queues offer FIFO service, and we assum…
In the present paper the infinite-server MMAPkGk queueing model with random resource vector of customers, marked MAP arrival and semi-Markov (SM) arrival of catastrophes is considered. The joint generating functions (PGF) of transient and stationary distributions of number of busy servers and numbers of different types served customers, as well as Laplace transformations (LT) of joint distributions of total accumulated resources in the model at moment and total accumulated resources of served customers during time interval are found. The basic differential and renewal equations for transient a…
This paper considers the Poisson equation for general state-space Markov chains in continuous time. The main purpose of this paper is to present specific bounds for the solutions of the Poisson equation for general state-space Markov chains. The solutions of the Poisson equation are unique in the sense that they are expressed in terms of a certain probabilistically interpretable solution (called the {\it standard solution}). Thus, we establish some specific bounds for the standard solution under the $f$-modulated drift condition (which is a kind of Foster-Lyapunov-type condition) and some mode…
We consider a Markovian single-server retrial queueing system with a constant retrial rate. Conditions of null ergodicity and exponential ergodicity for the correspondent process, as well as bounds on the rate of convergence are obtained.
A mean-field extension of the queueing system \(GI/GI/1\) is considered. The process is constructed as a Markov solution of a martingale problem. Uniqueness in distribution is established under a bit different sets of assumptions on intensities.
Donsker Theorem is perhaps the most famous invariance principle result for Markov processes. It states that when properly normalized, a random walk behaves asymptotically like a Brownian motion. This approach can be extended to general Markov processes whose driving parameters are taken to a limit, which can lead to insightful results in contexts like large distributed systems or queueing networks. The purpose of this paper is to assess the rate of convergence in these so-called diffusion approximations, in a queueing context. To this end, we extend the functional Stein method introduced for t…
Multi-class queueing networks (McQNs) extend the classical concept of Jackson network by allowing jobs of different classes to visit the same server. While such a generalization seems rather natural, from a structural perspective there is a significant gap between the two concepts. Nice analytical features of Jackson networks, such as stability conditions, product-form equilibrium distributions, and stochastic monotonicity do not immediately carry over to the multi-class framework. The aim of this paper is to shed some light on this structural gap, focusing on monotonicity properties. To this …
We prove the existence of limiting distributions for a large class of Markov chains on a general state space in a random environment. We assume suitable versions of the standard drift and minorization conditions. In particular, the system dynamics should be contractive on the average with respect to the Lyapunov function and large enough small sets should exist with large enough minorization constants. We also establish that a law of large numbers holds for bounded functionals of the process. Applications to queuing systems, to machine learning algorithms and to autoregressive processes are pr…
Let $\{\boldsymbol{X}_n\}$ be a discrete-time $d$-dimensional process on $\mathbb{Z}_+^d$ with a supplemental (background) process $\{J_n\}$ on a finite set and assume the joint process $\{\boldsymbol{Y}_n\}=\{(\boldsymbol{X}_n,J_n)\}$ to be Markovian. Then, the process $\{\boldsymbol{X}_n\}$ can be regarded as a kind of reflecting random walk (RRW for short) in which the transition probabilities of the RRW are modulated according to the state of the background process $\{J_n\}$; we assume this modulation is space-homogeneous inside $\mathbb{Z}_+^d$ and on each boundary face of $\mathbb{Z}_+^d…
In this paper the infinite server queue model in semi-Markov random environment with k Markov arrival streams, random resources of customers, and catastrophes is considered. After catastrophes occur, all customers in the model are flashed out and the system jumps into recovery station. After the recovery time the model works from the empty state. The transient and stationary joint distributions of numbers of different types of customers in the model at moment t, numbers of different types of served in some interval customers, volume of accumulated resources in the model at moment t, and total …
This paper discusses a maintenance network with failed items that can be removed, repaired, redistributed, and reused under two batch policies: one for removing the failed items from each base to a maintenance shop and the other for redistributing the repaired items from the maintenance shop to bases. This maintenance network can be considered a virtual closed queueing network, and the Markov system of each node is described as an elegant block-structured Markov process whose stationary probabilities can be computed by the RG-factorizations. The structure of this maintenance network is novel a…
As a favorite urban public transport mode, the bike sharing system is a large-scale and complicated system, and there exists a key requirement that a user and a bike should be matched sufficiently in time. Such matched behavior makes analysis of the bike sharing systems more difficult and challenging. This paper considers a more general large-scale bike sharing system from two important views: (a) Bikes move in an irreducible path graph, which is related to geographical structure of the bike sharing system; and (b) Markovian arrival processes (MAPs) are applied to describe the non-Poisson and …
Ever since Tassiulas and Ephremides (1992) proposed the maximum weight scheduling algorithm of throughput-optimality for constrained queueing networks that arise in the context of communication networks, extensive efforts have been devoted to resolving its most important drawback: high complexity. This paper proposes a generic framework for designing throughput- optimal and low-complexity scheduling algorithms for constrained queueing networks. Under our framework, a scheduling algorithm updates current schedules by interacting with a given oracle system that generates an approximate solution …
In this work, we focus on the stationary analysis of a specific class of continuous time Markov-modulated reflected random walks in the quarter plane with applications in the modelling of two-node Markov-modulated queueing networks with coupled queues. The transition rates of the two-dimensional process depend on the state of a finite state Markovian background process. Such a modulation is space homogeneous in the set of inner states of the two-dimensional lattice but may be different in the set of states at its boundaries. To obtain the stationary distribution, we apply the power series appr…
Motivated by queueing systems with heterogeneous parallel servers, we consider a class of structured multi-dimensional Markov processes whose state space can be partitioned into two parts: a finite set of boundary states and a structured multi-dimensional set of states, exactly one dimension of which is infinite. Using separation of variables, we show that the equilibrium distribution, typically of the queue length, can be represented as a linear combination of product forms. For an important subclass of queueing systems, we characterize explicitly the waiting time distribution in terms of mix…
We consider Markov processes, which describe e.g. queueing network processes, in a random environment which influences the network by determining random breakdown of nodes, and the necessity of repair thereafter. Starting from an explicit steady state distribution of product form available in the literature, we notice that this steady state distribution does not provide information about the correlation structure in time and space (over nodes). We study this correlation structure via one step correlations for the queueing-environment process. Although formulas for absolute values of these corr…
The models studied in the steady state involve two queues which are served either by a single server whose speed depends on the number of jobs present, or by several parallel servers whose number may be controlled dynamically. Job service times have a two-phase Coxian distribution and the second phase is given lower priority than the first. The trade-offs between holding costs and energy consumption costs are examined by means of a suitable cost functions. Two different two-dimensional Markov process are solved exactly. The solutions are used in several numerical experiments. Some counter-intu…
We study multiclass many-server queues for which the arrival, service and abandonment rates are all modulated by a common finite-state Markov process. We assume that the system operates in the "averaged" Halfin-Whitt regime, which means that it is critically loaded in the average sense, although not necessarily in each state of the Markov process. We show that under any static priority policy, the Markov-modulated diffusion-scaled queueing process is geometrically ergodic. This is accomplished by employing a solution to an associated Poisson equation in order to construct a suitable Lyapunov f…
The bike sharing systems are fast increasing as a public transport mode in urban short trips, and have been developed in many major cities around the world. A major challenge in the study of bike sharing systems is that large-scale and complex queueing networks have to be applied through multi-dimensional Markov processes, while their discussion always suffers a common difficulty: State space explosion. For this reason, this paper provides a mean-field computational method to study such a large-scale bike sharing steps: Firstly, a multi-dimensional Markov process is set up for expressing the s…
We prove that the long term distribution of the queue length process in an ergodic generalised Jackson network obeys the Large Deviation Principle with a deviation function given by the quasipotential. The latter is related to the unique long term idempotent distribution, which is also a stationary idempotent distribution, of the large deviation limit of the queue length processes. The proof draws on developments in queueing network stability and idempotent probability.
We develop a robust queueing network analyzer algorithm to approximate the steady-state performance of a single-class open queueing network of single-server queues with Markovian routing. The algorithm allows non-renewal external arrival processes, general service-time distributions and customer feedback. We focus on the customer flows, defined as the continuous-time processes counting customers flowing into or out of the network, or flowing from one queue to another. Each flow is partially characterized by its rate and a continuous function that measures the stochastic variability over time. …
The Multi-class Queueing Network (McQN) arises as a natural multi-class extension of the traditional (single-class) Jackson network. In a single-class network subcriticality (i.e. subunitary nominal workload at every station) entails stability, but this is no longer sufficient when jobs/customers of different classes (i.e. with different service requirements and/or routing scheme) visit the same server; therefore, analytical conditions for stability of McQNs are lacking, in general. In this note we design a numerical (simulation-based) method for determining the stability region of a McQN, in …
We consider a semi-open queueing network (SOQN), where a customer requires exactly one resource from the resource pool for service. If there is a resource available, the customer is immediately served and the resource enters an inner network. If there is no resource available, the new customer has to wait in an external queue until one becomes available ("backordering"). When a resource exits the inner network, it is returned to the resource pool and waits for another customer. In this paper, we present a new solution approach. To approximate the inner network with the resource pool of the SOQ…
Shenzhen Research Institute of Big Data（深圳大数据研究院）；The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））；Cornell University（康奈尔大学）
Novel advanced policy gradient (APG) methods, such as Trust Region policy optimization and Proximal policy optimization (PPO), have become the dominant reinforcement learning algorithms because of their ease of implementation and good practical performance. A conventional setup for notoriously difficult queueing network control problems is a Markov decision problem (MDP) that has three features: infinite state space, unbounded costs, and long-run average cost objective. We extend the theoretical framework of these APG methods for such MDP problems. The resulting PPO algorithm is tested on a pa…
The single server queue with multiple customer types and semi-Markovian service times, sometimes referred to as the $M/SM/1$ queue, has been well-studied since its introduction by Neuts in 1966. In this paper, we apply an extension of this model, with batch arrivals and exceptional first service, to road traffic situations involving multiple streams of conflicting traffic. In particular, we use it in the context of gap acceptance models where low-priority traffic needs to cross (or, depending on the application, merge with) another traffic flow of higher priority. Traditionally, gap acceptance…
