One of the key performance measures in queueing systems is the exponential decay rate of the steady-state tail probabilities of the queue lengths. It is known that if a corresponding fluid model is stable and the stochastic primitives have finite moments, then the queue lengths also have finite moments, so that the tail probability \pr(\cdot >s) decays faster than s^{-n} for any n. It is natural to conjecture that the decay rate is in fact exponential.
In this paper an example is constructed to demonstrate that this conjecture is false. For a specific stationary policy applied to a network with exponentially distributed interarrival and service times it is shown that the corresponding fluid limit model is stable, but the tail probability for the buffer length decays slower than s^{-\log s}.
This paper addresses the question when the underlying Markov process of a multiclass queueing network is positive Harris recurrent. It is well-known that stability of the fluid limit model is a sufficient condition for this. Hence, stability of fluid (limit) models is of vital interest. Recently, it has been shown that if the fluid model satisfies certain properties, it is stable if and only if there exists a Lyapunov function. In this paper a new method is provided to conclude that the underlying Markov process is positive Harris recurrent if the fluid model is stable by using explicitly the …
We consider multiclass feedforward queueing networks with first in first out and priority service disciplines at the nodes, and class dependent deterministic routing between nodes. The random behavior of the network is constructed from cumulative arrival and service time processes which are assumed to satisfy an appropriate sample path large deviation principle. We establish logarithmic asymptotics of large deviations for waiting time, idle time, queue length, departure and sojourn-time processes in critical loading. This transfers similar results from Puhalskii about single class queueing net…
We consider a Markovian single-server retrial queueing system with a constant retrial rate. Conditions of null ergodicity and exponential ergodicity for the correspondent process, as well as bounds on the rate of convergence are obtained.
We study a differential game that governs the moderate-deviation heavy-traffic asymptotics of a multiclass single-server queueing control problem with a risk-sensitive cost. We consider a cost set on a finite but sufficiently large time horizon, and show that this formulation leads to stationary feedback policies for the game. Several aspects of the game are explored, including its characterization via a (one-dimensional) free boundary problem, the semi-explicit solution of an optimal strategy, and the specification of a saddle point. We emphasize the analogy to the well-known Harrison-Taksar …
This paper studies a scheduling control problem for a single-server multiclass queueing network in heavy traffic, operating in a changing environment. The changing environment is modeled as a finite state Markov process that modulates the arrival and service rates in the system. Various cases are considered: fast changing environment, fixed environment and slow changing environment. In each of the cases, using weak convergence analysis, in particular functional limit theorems for renewal processes and ergodic Markov processes, it is shown that an appropriate "averaged" version of the classical…
We study a multiclass M/M/1 queueing control problem with finite buffers under heavy-traffic where the decision maker is uncertain about the rates of arrivals and service of the system and by scheduling and admission/rejection decisions acts to minimize a discounted cost that accounts for the uncertainty. The main result is the asymptotic optimality of a $cμ$-type of policy derived via underlying stochastic differential games studied in [16]. Under this policy, with high probability, rejections are not performed when the workload lies below some cut-off that depends on the ambiguity level. Whe…
A multi-class single-server queueing model with finite buffers, in which scheduling and admission of customers are subject to control, is studied in the moderate deviation heavy traffic regime. A risk-sensitive cost set over a finite time horizon $[0,T]$ is considered. The main result is the asymptotic optimality of a control policy derived via an underlying differential game. The result is the first to address a queueing control problem at the moderate deviation regime that goes beyond models having the so called pathwise minimality property. Moreover, despite the well known fact that an opti…
In this paper, we present counter-intuitive examples for the multiclass queueing network system. In the system, each station may serve more than one job class with differentiated service priority, and each job may require service sequentially by more than one service station. In our examples, the network performance is improved even when more workloads are admitted for service.
We analyze an interacting queueing network on $\mathbb{Z}^d$ that was introduced in Sankararaman-Baccelli-Foss (2019) as a model for wireless networks. We show that the marginals of the minimal stationary distribution have exponential tails. This is used to furnish asymptotics for the maximum steady state queue length in growing boxes around the origin. We also establish a decay of correlations which shows that the minimal stationary distribution is strongly mixing, and hence, ergodic with respect to translations on $\mathbb{Z}^d$.
We introduce a multiclass single-server queueing system in which the arrival rates depend on the current job in service. The system is characterized by a matrix of arrival rates in lieu of a vector of arrival rates. Our proposed model departs from existing state-dependent queueing models in which the parameters depend primarily on the number of jobs in the system rather than on the job in service. We formulate the queueing model and its corresponding fluid model and proceed to obtain the necessary and sufficient conditions for stability via fluid models. Utilizing the natural connection with t…
The Multi-class Queueing Network (McQN) arises as a natural multi-class extension of the traditional (single-class) Jackson network. In a single-class network subcriticality (i.e. subunitary nominal workload at every station) entails stability, but this is no longer sufficient when jobs/customers of different classes (i.e. with different service requirements and/or routing scheme) visit the same server; therefore, analytical conditions for stability of McQNs are lacking, in general. In this note we design a numerical (simulation-based) method for determining the stability region of a McQN, in …
We consider a multidimentional Brownian control problem (BCP) with model uncertainty that formally emerges from a multiclass M/M/1 queueing control problem under heavy-traffic with model uncertainty. The BCP is formulated as a multidimensional stochastic differential game with two players: a minimizer that has an equivalent role to the decision maker in the queueing control problem and a maximizer whose role is to set up the uncertainty of the model. The dynamics are driven by a Brownian motion. We show that a state-space collapse propery holds. That is, the multidimensional BCP can be reduced…
AM/M/N+Mqueueingnetworkisconsideredwithdindependentcustomerclasses and d server pools in Halfin-Whitt regime. Class i customers has priority for service in pool i for i = 1, . . . , d, and may access some other pool if the pool has an idle server and all the servers in pool i are busy. We formulate an ergodic control problem where the running cost is given by a non- negative convex function with polynomial growth. We show that the limiting controlled diffusion is modeled by an action space which depends on the state variable. We provide a complete analysis for the limiting ergodic control prob…
We consider the stability of robust scheduling policies for multiclass queueing networks. These are open networks with arbitrary routing matrix and several disjoint groups of queues in which at most one queue can be served at a time. The arrival and potential service processes and routing decisions at the queues are independent, stationary and ergodic. A scheduling policy is called robust if it does not depend on the arrival and service rates nor on the routing probabilities. A policy is called throughput-optimal if it makes the system stable whenever the parameters are such that the system ca…
We study the ergodic properties of a class of multidimensional piecewise Ornstein-Uhlenbeck processes with jumps, which contains the limit of the queueing processes arising in multiclass many-server queues with heavy-tailed arrivals and/or asymptotically negligible service interruptions in the Halfin-Whitt regime as special cases. In these queueing models, the Itô equations have a piecewise linear drift, and are driven by either (1) a Brownian motion and a pure-jump Lévy process, or (2) an anisotropic Lévy process with independent one-dimensional symmetric $α$-stable components, or (3) an anis…
One of the basic properties of a queueing network is stability. Roughly speaking, it is the property that the total number of jobs in the network remains bounded as a function of time. One of the key questions related to the stability issue is how to determine the exact conditions under which a given queueing network operating under a given scheduling policy remains stable. While there was much initial progress in addressing this question, most of the results obtained were partial at best and so the complete characterization of stable queueing networks is still lacking. In this paper, we resol…
We use simulation to estimate the steady-state performance of a stable multiclass queueing network. Standard estimators have been seen to perform poorly when the network is heavily loaded. We introduce two new simulation estimators. The first provides substantial variance reductions in moderately-loaded networks at very little additional computational cost. The second estimator provides substantial variance reductions in heavy traffic, again for a small additional computational cost. Both methods employ the variance reduction method of control variates, and differ in terms of how the control v…
We consider a heterogeneous queueing system consisting of one large pool of $O(r)$ identical servers, where $r\to\infty$ is the scaling parameter. The arriving customers belong to one of several classes which determines the service times in the distributional sense. The system is heavily loaded in the Halfin-Whitt sense, namely the nominal utilization is $1-a/\sqrt{r}$ where $a>0$ is the spare capacity parameter. Our goal is to obtain bounds on the steady state performance metrics such as the number of customers waiting in the queue $Q^r(\infty)$. While there is a rich literature on deriving p…
A classical result for the steady-state queue-length distribution of single-class queueing systems is the following: the distribution of the queue length just before an arrival epoch equals the distribution of the queue length just after a departure epoch. The constraint for this result to be valid is that arrivals, and also service completions, with probability one occur individually, i.e., not in batches. We show that it is easy to write down somewhat similar balance equations for {\em multidimensional} queue-length processes for a quite general network of multiclass multiserver queues. We f…
Multiclass open queueing networks find wide applications in communication, computer and fabrication networks. Often one is interested in steady-state performance measures associated with these networks. Conceptually, under mild conditions, a regenerative structure exists in multiclass networks, making them amenable to regenerative simulation for estimating the steady-state performance measures. However, typically, identification of a regenerative structure in these networks is difficult. A well known exception is when all the interarrival times are exponentially distributed, where the instants…
We study multiclass many-server queues for which the arrival, service and abandonment rates are all modulated by a common finite-state Markov process. We assume that the system operates in the "averaged" Halfin-Whitt regime, which means that it is critically loaded in the average sense, although not necessarily in each state of the Markov process. We show that under any static priority policy, the Markov-modulated diffusion-scaled queueing process is geometrically ergodic. This is accomplished by employing a solution to an associated Poisson equation in order to construct a suitable Lyapunov f…
This paper considers a parallel system of queues fed by independent arrival streams, where the service rate of each queue depends on the number of customers in all of the queues. Necessary and sufficient conditions for the stability of the system are derived, based on stochastic monotonicity and marginal drift properties of multiclass birth and death processes. These conditions yield a sharp characterization of stability for systems, where the service rate of each queue is decreasing in the number of customers in other queues, and has uniform limits as the queue lengths tend to infinity. The r…
In this paper, we study optimal control problems for multiclass GI/M/n+M queues in an alternating renewal (up-down) random environment in the Halfin-Whitt regime. Assuming that the downtimes are asymptotically negligible and only the service processes are affected, we show that the limits of the diffusion-scaled state processes under non-anticipative, preemptive, work-conserving scheduling policies, are controlled jump diffusions driven by a compound Poisson jump process. We establish the asymptotic optimality of the infinite-horizon discounted and long-run average (ergodic) problems for the q…
As a favorite urban public transport mode, the bike sharing system is a large-scale and complicated system, and there exists a key requirement that a user and a bike should be matched sufficiently in time. Such matched behavior makes analysis of the bike sharing systems more difficult and challenging. This paper considers a more general large-scale bike sharing system from two important views: (a) Bikes move in an irreducible path graph, which is related to geographical structure of the bike sharing system; and (b) Markovian arrival processes (MAPs) are applied to describe the non-Poisson and …
We study the ergodic properties of a class of controlled stochastic differential equations (SDEs) driven by $α$-stable processes which arise as the limiting equations of multiclass queueing models in the Halfin-Whitt regime that have heavy-tailed arrival processes. When the safety staffing parameter is positive, we show that the SDEs are uniformly ergodic and enjoy a polynomial rate of convergence to the invariant probability measure in total variation, which is uniform over all stationary Markov controls resulting in a locally Lipschitz continuous drift. We also derive a matching lower bound …
We study weak ergodicity, bounds on the rate of convergence, and problems of computing of the limiting characteristics for an inhomogeneous $M_t|M_t|S$ queueing model with possible catastrophes.
Consider a single server system serving a multiclass population. Some popular scheduling policies for such system are the discriminatory processor sharing (DPS), discriminatory random order service (DROS), generalized processor sharing (GPS) and weighted fair queueing (WFQ). In this paper, we propose two classes of policies, namely MPS (multiclass processor sharing) and MROS (multiclass random order service), that generalize the four policies mentioned above. For the special case when the multiclass population arrive according to Poisson processes and have independent and exponential service r…
We consider the discrete Hammersley-Aldous-Diaconis process (HAD) and the totally asymmetric simple exclusion process (TASEP) in Z. The basic coupling induces a multiclass process which is useful in discussing shock measures and other important properties of the processes. The invariant measures of the multiclass systems are the same for both processes, and can be constructed as the law of the output process of a system of multiclass queues in tandem; the arrival and service processes of the queueing system are a collection of independent Bernoulli product measures. The proof of invariance inv…
We consider the so-called GI/GI/N queueing network in which a stream of jobs with independent and identically distributed service times arrive according to a renewal process to a common queue served by $N$ identical servers in a First-Come-First-Serve manner. We introduce a two-component infinite-dimensional Markov process that serves as a diffusion model for this network, in the regime where the number of servers goes to infinity and the load on the network scales as $1 - βN^{-1/2}+ o(N^{-1/2})$ for some $β> 0$. Under suitable assumptions, we characterize this process as the unique solution t…
Shenzhen Research Institute of Big Data（深圳大数据研究院）；The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））；Cornell University（康奈尔大学）
Novel advanced policy gradient (APG) methods, such as Trust Region policy optimization and Proximal policy optimization (PPO), have become the dominant reinforcement learning algorithms because of their ease of implementation and good practical performance. A conventional setup for notoriously difficult queueing network control problems is a Markov decision problem (MDP) that has three features: infinite state space, unbounded costs, and long-run average cost objective. We extend the theoretical framework of these APG methods for such MDP problems. The resulting PPO algorithm is tested on a pa…
In this paper we revisit the Markovian queueing system with a single server, infinite capacity queue and the special queue skipping policy. Customers arrive in batches, but are served one by one according to any conservative discipline. The size of the arriving batch becomes known upon its arrival and at any time instant the total number of customers in the system is also known. According to the adopted queue skipping policy if a batch, which size is greater than the current system size, arrives to the system, all current customers in the system are removed from it and the new batch is placed …
We establish a large deviation principle for the empirical measure process associated with a general class of finite-state mean field interacting particle systems with Lipschitz continuous transition rates that satisfy a certain ergodicity condition. The approach is based on a variational representation for functionals of a Poisson random measure. Under an appropriate strengthening of the ergodicity condition, we also prove a locally uniform large deviation principle. The main novelty is that more than one particle is allowed to change its state simultaneously, and so a standard approach to th…
