We study a many-server queueing model with server vacations, where the population size dynamics of servers and customers are coupled: a server may leave for vacation only when no customers await, and the capacity available to customers is directly affected by the number of servers on vacation. We focus on scaling regimes in which server dynamics and queue dynamics fluctuate at matching time scales, so that their limiting dynamics are coupled. Specifically, we argue that interesting coupled dynamics occur in (a) the Halfin-Whitt regime, (b) the nondegenerate slowdown regime, and (c) the intermediate, near Halfin-Whitt regime; whereas the dynamics asymptotically decouple in the other heavy traffic regimes. We characterize the limiting dynamics, which are different for each scaling regime. We consider relevant respective performance measures for regimes (a) and (b) --- namely, the probability of wait and the slowdown. While closed form formulas for these performance measures have been derived for models that do not accommodate server vacations, it is difficult to obtain closed form formulas for these performance measures in the setting with server vacations. Instead, we propose formulas that approximate these performance measures, and depend on the steady-state mean number of available servers and previously derived formulas for models without server vacations. We test the accuracy of these formulas numerically.
In this paper, we consider systems that can be modelled by $M \mid M \mid n$ queues with heterogeneous servers and non informed customers. Considering any two servers: we show that the probability that the fastest server is busy is smaller than the probability that the slowest server is busy. Moreover, we show that the effective rate of service done by the fastest server is larger than effective rate of service done by the slowest server.
We consider a service system model primarily motivated by the problem of efficient assignment of virtual machines to physical host machines in a network cloud, so that the number of occupied hosts is minimized. There are multiple types of arriving customers, where a customer's mean service time depends on its type. There is an infinite number of servers. Multiple customers can be placed for service into one server, subject to general "packing" constraints. Service times of different customers are independent, even if served simultaneously by the same server. Each new arriving customer is place…
Customer slowdown describes the phenomenon that a customer's service requirement increases with experienced delay. In healthcare settings, there is substantial empirical evidence for slowdown, particularly when a patient's delay exceeds a certain threshold. For such threshold slowdown situations, we design and analyze a many-server system that leads to a two-dimensional Markov process. Analysis of this system leads to insights into the potentially detrimental effects of slowdown, especially in heavy-traffic conditions. We quantify the consequences of underprovisioning due to neglecting slowdow…
The model is a service system, consisting of several large server pools. A server processing speed and buffer size (which may be finite or infinite) depend on the pool. The input flow of customers is split equally among a fixed number of routers, which must assign customers to the servers immediately upon arrival. We consider an asymptotic regime in which the customer total arrival rate and pool sizes scale to infinity simultaneously, in proportion to a scaling parameter $n$, while the number of routers remains fixed. We define and study a multi-router generalization of the pull-based customer…
This paper studies multiclass loss systems with two layers of servers, where each server at the first layer is dedicated to a certain customer class, while the servers at the second layer can handle all customer classes. The routing of customers follows an overflow scheme, where arriving customers are preferentially directed to the first layer. Stochastic comparison and coupling techniques are developed for studying how the system is affected by packing of customers, altered service rates, and altered server configurations. This analysis leads to easily computable upper and lower bounds for th…
In this paper, equilibrium strategies and optimal balking strategies of customers in a constant retrial queue with multiple vacations and the $N$-policy under two information levels, respectively, are investigated. We assume that there is no waiting area in front of the server and an arriving customer is served immediately if the server is idle; otherwise (the server is either busy or on a vacation) it has to leave the system to join a virtual retrial orbit waiting for retrials according to the FCFS rules. After a service completion, if the system is not empty, the server becomes idle, availab…
A service system with multiple types of arriving customers is considered. There is an infinite number of homogeneous servers. Multiple customers can be placed for simultaneous service into one server, subject to general packing constraints. Each new arriving customer is placed for service immediately, either into an occupied server, as long as packing constraints are not violated, or into an empty server. After service completion, each customer leaves its server and the system. The basic objective is to minimize the number of occupied servers in steady state. We study a Greedy-Random (GRAND) p…
We consider three parallel service models in which customers of several types are served by several types of servers subject to a bipartite compatibility graph, and the service policy is first come first served. Two of the models have a fixed set of servers. The first is a queueing model in which arriving customers are assigned to the longest idling compatible server if available, or else queue up in a single queue, and servers that become available pick the longest waiting compatible customer, as studied by Adan and Weiss, 2014. The second is a redundancy service model where arriving customer…
We consider a Markovian queue subject to Poisson generated catastrophes. Whenever a catastrophe occurs, all customers are forced to abandon the system, the server is rendered inoperative and an exponential repair time is set on. We assume that the arriving customers decide whether to join the system or balk, based on a natural reward-cost structure. We study the balking behavior of the customers and derive the corresponding Nash equilibrium strategies.
The shortest-remaining-processing-time (SRPT) scheduling policy has been extensively studied, for more than 50 years, in single-server queues with infinitely patient jobs. Yet, much less is known about its performance in multiserver queues. In this paper, we present the first theoretical analysis of SRPT in multiserver queues with abandonment. In particular, we consider the M/GI/s+GI queue and demonstrate that, in the many-sever overloaded regime, performance in the SRPT queue is equivalent, asymptotically in steady state, to a preemptive two-class priority queue where customers with short ser…
This paper considers a discrete-time single-server queue with a single acceptance period for a Poissonian population of homogeneous customers. Customers are served on a first-come first-served (FCFS) basis, and their service times are independent and identically distributed with a general distribution. We assume that each customer chooses her/his arrival-time slot with the goal of minimizing her/his expected waiting time in competition with other customers. For this queueing game, we derive a symmetric (mixed-strategy) Nash equilibrium; that is, an equilibrium arrival-time distribution of homo…
We use multidimensional diffusion processes to approximate the dynamics of a queue served by many parallel servers. The queue is served in the first-in-first-out (FIFO) order and the customers waiting in queue may abandon the system without service. Two diffusion models are proposed in this paper. They differ in how the patience time distribution is built into them. The first diffusion model uses the patience time density at zero and the second one uses the entire patience time distribution. To analyze these diffusion models, we develop a numerical algorithm for computing the stationary distri…
A parallel server system with $n$ identical servers is considered. The service time distribution has a finite mean $1/μ$, but otherwise is arbitrary. Arriving customers are be routed to one of the servers immediately upon arrival. Join-Idle-Queue routing algorithm is studied, under which an arriving customer is sent to an idle server, if such is available, and to a randomly uniformly chosen server, otherwise. We consider the asymptotic regime where $n\to\infty$ and the customer input flow rate is $λn$. Under the condition $λ/μ<1/2$, we prove that, as $n\to\infty$, the sequence of (appropriatel…
A service system with multiple types of customers, arriving according to Poisson processes, is considered. The system is heterogeneous in that the servers also can be of multiple types. Each customer has an independent exponentially distributed service time, with the mean determined by its type. Multiple customers (possibly of different types) can be placed for service into one server, subject to "packing" constraints, which depend on the server type. Service times of different customers are independent, even if served simultaneously by the same server. The large-scale asymptotic regime is con…
We study a parallel queueing system with multiple types of servers and customers. A bipartite graph describes which pairs of customer-server types are compatible. We consider the service policy that always assigns servers to the first, longest waiting compatible customer, and that always assigns customers to the longest idle compatible server if on arrival, multiple compatible servers are available. For a general renewal stream of arriving customers and general service time distributions, the behavior of such systems is very complicated. In particular, the calculation of matching rates, the fr…
We consider a stochastic bipartite matching model consisting of multi-class customers and multi-class servers. Compatibility constraints between the customer and server classes are described by a bipartite graph. Each time slot, exactly one customer and one server arrive. The incoming customer (resp. server) is matched with the earliest arrived server (resp. customer) with a class that is compatible with its own class, if there is any, in which case the matched customer-server couple immediately leaves the system; otherwise, the incoming customer (resp. server) waits in the system until it is …
The focus of this paper is on the asymptotics of large-time numbers of customers in time-periodic Markovian many-server queues with customer abandonment in heavy traffic. Limit theorems are obtained for the periodic number-of-customers processes under the fluid and diffusion scalings. Other results concern limits for general time-dependent queues and for time-homogeneous queues in steady state.
This paper presents a heavy traffic analysis of the behavior of multi-class acyclic queueing networks in which the customers have deadlines. We assume the queueing system consists of J stations, and there are K different customer classes. Customers from each class arrive to the network according to independent renewal processes. The customers from each class are assigned a random deadline drawn from a deadline distribution associated with that class and they move from station to station according to a fixed acyclic route. The customers at a given node are processed according to the earliest-de…
We study many-server queues with abandonment in which customers have general service and patience time distributions. The dynamics of the system are modeled using measure- valued processes, to keep track of the residual service and patience times of each customer. Deterministic fluid models are established to provide first-order approximation for this model. The fluid model solution, which is proved to uniquely exists, serves as the fluid limit of the many-server queue, as the number of servers becomes large. Based on the fluid model solution, first-order approximations for various performance…
Motivated by the ongoing COVID-19 pandemic, this paper investigates customers' infection risk by evaluating the overlapping time of a virtual customer with others in queueing systems. Most of the current methodologies focus on characterizing the risk in stationary systems, which may not apply to the more practical time-varying systems. As such, we propose an approximation framework that relies on the fluid limit to compute the expected overlapping time in time-varying queueing systems. Simulation experiments verify the accuracy of our approach.
The paper studies a closed queueing network containing two types of node. The first type (server station) is an infinite server queueing system, and the second type (client station) is a single server queueing system with autonomous service, i.e. every client station serves customers (units) only at random instants generated by strictly stationary and ergodic sequence of random variables. It is assumed that there are $r$ server stations. At the initial time moment all units are distributed in the server stations, and the $i$th server station contains $N_i$ units, $i=1,2,...,r$, where all the v…
We consider the problem of scheduling a queueing system in which many statistically identical servers cater to several classes of impatient customers. Service times and impatience clocks are exponential while arrival processes are renewal. Our cost is an expected cumulative discounted function, linear or nonlinear, of appropriately normalized performance measures. As a special case, the cost per unit time can be a function of the number of customers waiting to be served in each class, the number actually being served, the abandonment rate, the delay experienced by customers, the number of idli…
A single server retrial queueing system with two-classes of orbiting customers, and general class dependent service times is considered. If an arriving customer finds the server unavailable, it enters a virtual queue, called the orbit, according to its type. The customers from the orbits retry independently to access the server according to the constant retrial policy. We derive the generating function of the stationary distribution of the number of orbiting customers at service completion epochs in terms of the solution of a Riemann boundary value problem. For the symmetrical system we also d…
This paper studies the heavy-traffic (HT) behaviour of queueing networks with a single roving server. External customers arrive at the queues according to independent renewal processes and after completing service, a customer either leaves the system or is routed to another queue. This type of customer routing in queueing networks arises very naturally in many application areas (in production systems, computer- and communication networks, maintenance, etc.). In these networks, the single most important characteristic of the system performance is oftentimes the path time, i.e. the total time sp…
We consider a single-server GI/GI/1 queueing system with feedback. We assume the service times distribution to be (intermediate) regularly varying. We find the tail asymptotics for a customer's sojourn time in two regimes: the customer arrives in an empty system, and the customer arrives in the system in the stationary regime. In particular, in the case of Poisson input we use the branching processes structure and provide more precise formulae. As auxiliary results, we find the tail asymptotics for the busy period distribution in a single-server queue with an intermediate varying service times…
We consider GI/Ph/n+M parallel-server systems with a renewal arrival process, a phase-type service time distribution, n homogenous servers, and an exponential patience time distribution with positive rate. We show that in the Halfin-Whitt regime, the sequence of stationary distributions corresponding to the normalized state processes is tight. As a consequence, we establish an interchange of heavy traffic and steady state limits for GI/Ph/n+M queues.
In this paper the infinite server queue model in semi-Markov random environment with k Markov arrival streams, random resources of customers, and catastrophes is considered. After catastrophes occur, all customers in the model are flashed out and the system jumps into recovery station. After the recovery time the model works from the empty state. The transient and stationary joint distributions of numbers of different types of customers in the model at moment t, numbers of different types of served in some interval customers, volume of accumulated resources in the model at moment t, and total …
We study a system, where a random flow of customers is served by servers (called agents) invited on-demand. Each invited agent arrives into the system after a random time; after each service completion, an agent returns to the system or leaves it with some fixed probabilities. Customers and/or agents may be impatient, that is, while waiting in queue, they leave the system at a certain rate (which may be zero). We consider the queue-length-based feedback scheme, which controls the number of pending agent invitations, depending on the customer and agent queue lengths and their changes. The basic…
In this paper we study the number of customers in infinite-server queues with a self-exciting (Hawkes) arrival process. Initially we assume that service requirements are exponentially distributed and that the Hawkes arrival process is of a Markovian nature. We obtain a system of differential equations that characterizes the joint distribution of the arrival intensity and the number of customers. Moreover, we provide a recursive procedure that explicitly identifies (transient and stationary) moments. Subsequently, we allow for non-Markovian Hawkes arrival processes and non-exponential service t…
This paper examines the performance of multi-class multi-server bipartite queueing systems under a FCFS-ALIS service discipline, where each arriving customer is only compatible with a subset of servers. We analyze the system under conventional heavy-traffic conditions, where the traffic intensity approaches one from below. Building upon the formulation and results of Afeche et al. 2021, we generalize the model by allowing the vector of arrival rates to approach the heavy-traffic limit from an arbitrary direction. We characterize the steady-state waiting times of the various customer classes an…
Motivated by a web-server model, we present a queueing network consisting of two layers. The first layer incorporates the arrival of customers at a network of two single-server nodes. We assume that the inter-arrival and the service times have general distributions. Customers are served according to their arrival order at each node and after finishing their service they can re-enter at nodes several times (as new customers) for new services. At the second layer, active servers act as jobs which are served by a single server working at speed one in a Processor-Sharing fashion. We further assume…
In the present paper the infinite-server MMAPkGk queueing model with random resource vector of customers, marked MAP arrival and semi-Markov (SM) arrival of catastrophes is considered. The joint generating functions (PGF) of transient and stationary distributions of number of busy servers and numbers of different types served customers, as well as Laplace transformations (LT) of joint distributions of total accumulated resources in the model at moment and total accumulated resources of served customers during time interval are found. The basic differential and renewal equations for transient a…
Given a random variable $N$ with values in ${\mathbb{N}}$, and $N$ i.i.d. positive random variables $\{μ_k\}$, we consider a queue with renewal arrivals and $N$ exponential servers, where server $k$ serves at rate $μ_k$, under two work conserving routing schemes. In the first, the service rates $\{μ_k\}$ need not be known to the router, and each customer to arrive at a time when some servers are idle is routed to the server that has been idle for the longest time (or otherwise it is queued). In the second, the service rates are known to the router, and a customer that arrives to find idle serv…
Following recent developments in the application of Stein's method in queueing theory, this paper is intended to be a short treatment showing how Stein's method can be developed and applied to the single server queue in heavy traffic. Here we provide two approaches to this approximation: one based on equilibrium couplings and another involving comparison of generators.
We establish a heavy-traffic limit theorem on convergence in distribution for the number of customers in a many-server queue when the number of servers tends to infinity. No critical loading condition is assumed. Generally, the limit process does not have trajectories in the Skorohod space. We give conditions for the convergence to hold in the topology of compact convergence. Some new results for an infinite server are also provided.
Consider the workload process for a single server queue with deterministic service times in which customers arrive according to a scheduled traffic process. A scheduled arrival sequence is one in which customers are scheduled to arrive at constant interarrival times, but each customer actual arrival time is perturbed from her scheduled arrival time by a random perturbation. In this paper, we consider a critically loaded queue in which the service rate equals the arrival rate. Unlike a queue fed by renewal traffic, this queue can be stable even in the presence of critical loading. We identify a…
This paper introduces and analyzes the notion of throughput suboptimality for many-server queueing systems in heavy traffic. The queueing model under consideration has multiple customer classes, indexed by a finite set $\mathcal{I}$, and heterogenous, exponential servers. Servers are dynamically chosen to serve customers, and buffers are available for customers waiting to be served. The arrival rates and the number of servers are scaled up in such a way that the processes representing the number of class-$i$ customers in the system, $i\in\mathcal{I}$, fluctuate about a static fluid model, that…
We consider general large-scale service systems with multiple customer classes and multiple server (agent) pools, mean service times depend both on the customer class and server pool. It is assumed that the allowed activities (routing choices) form a tree (in the graph with vertices being both customer classes and server pools). We study the behavior of the system under a natural (load balancing) routing/scheduling rule, Longest-Queue Freest-Server (LQFS-LB), in the many-server asymptotic regime, such that the exogenous arrival rates of the customer classes, as well as the number of agents in …
The model is motivated by the problem of load distribution in large-scale cloud-based data processing systems. We consider a heterogeneous service system, consisting of multiple large server pools. The pools are different in that their servers may have different processing speed and/or different buffer sizes (which may be finite or infinite). We study an asymptotic regime in which the customer arrival rate and pool sizes scale to infinity simultaneously, in proportion to some scaling parameter $n$. Arriving customers are assigned to the servers by a "router", according to a {\em pull-based} al…
In this paper we consider a single-server, cyclic polling system with switch-over times. A distinguishing feature of the model is that the rates of the Poisson arrival processes at the various queues depend on the server location. For this model we study the joint queue length distribution at polling epochs and at server's departure epochs. We also study the marginal queue length distribution at arrival epochs, as well as at arbitrary epochs (which is not the same in general, since we cannot use the PASTA property). A generalised version of the distributional form of Little's law is applied to…
We present a heavy traffic analysis of a single-server polling model, with the special features of retrials and glue periods. The combination of these features in a polling model typically occurs in certain optical networking models, and in models where customers have a reservation period just before their service period. Just before the server arrives at a station there is some deterministic glue period. Customers (both new arrivals and retrials) arriving at the station during this glue period will be served during the visit of the server. Customers arriving in any other period leave immediat…
The interest to retrial queueing systems is due to their application to telephone systems. The paper studies multiserver retrial queueing systems with $n$ servers. Arrival process is a quite general point process. An arriving customer occupies one of free servers. If upon arrival all servers are busy, then the customer waits for his service in orbit, and after random time retries more and more to occupy a server. The orbit has one waiting space only, and arriving customer, who finds all servers busy and the waiting space occupied, losses from the system. Time intervals between possible retrial…
A many-server queueing system is considered in which customers with independent and identically distributed service times enter service in the order of arrival. The state of the system is represented by a process that describes the total number of customers in the system, as well as a measure-valued process that keeps track of the ages of customers in service, leading to a Markovian description of the dynamics. Under suitable assumptions, a functional central limit theorem is established for the sequence of (centered and scaled) state processes as the number of servers goes to infinity. The li…
We consider a service system model primarily motivated by the problem of efficient assignment of virtual machines to physical host machines in a network cloud, so that the number of occupied hosts is minimized. There are multiple input flows of different type customers, with a customer mean service time depending on its type. There is infinite number of servers. A server packing {\em configuration} is the vector $k=\{k_i\}$, where $k_i$ is the number of type $i$ customers the server "contains". Packing constraints must be observed, namely there is a fixed finite set of configurations $k$ that …
This paper is devoted to the study of the number of customers in infinite-server systems driven by Hawkes processes. In these systems, the self-exciting arrival process is assumed to be represented by a Hawkes process and the self-exciting service process by a state-dependent Hawkes process (sdHawkes process). Under some suitable conditions, for the Hawkes/sdHawkes/infty system, the Markov property of the system is derived. The joint time-dependent distribution of the number of customers in the system, the arrival intensity and the server intensity is characterised by a system of differential …
We consider a large-scale flexible service system with two large server pools and two types of customers. Servers in pool 1 can only serve type 1 customers, while server in pool 2 are flexible -- they can serve both types 1 and 2. (This is a so-called "N-system." Our results hold for a more general class of systems as well.) The service rate of a customer depends both on its type and the pool where it is served. We study a priority service discipline, where type 2 has priority in pool 2, and type 1 prefers pool 1. We consider the Halfin-Whitt asymptotic regime, where the arrival rate of custom…
In this thesis, we study the queueing systems with heterogeneous servers and service rate uncertainty under the Halfin-Whitt heavy traffic regime. First, we analyse many server queues with abandonments when service rates are i.i.d. random variables. We derive a diffusion approximation using a novel method. The diffusion has a random drift, and hence depending on the realisations of service rates, the system can be in Quality Driven (QD), Efficiency Driven (ED) or Quality-Efficiency-Driven (QED) regime. When the system is under QD or QED regime, the abandonments are negligible in the fluid limi…
Imagine, you enter a grocery store to buy food. How many peopledo you overlap with in this store? How much time do you overlap witheach person in the store? In this paper, we answer these questions bystudying the overlap times between customers in the infinite serverqueue. We compute in closed form the steady state distribution ofthe overlap time between a pair of customers and the distribution ofthe number of customers that an arriving customer will overlap with.Finally, we define a residual process that counts the number of over-lapping customers that overlap in the queue for at leastδtime u…
The infinite-server queueing models with homogeneous and non-homogeneous arrivals of customers and catastrophes are considered. The probability generating functions of joint distributions of numbers of busy servers and served customers, as well as the Laplace-Stieltjes Transforms of distribution of busy period and distribution of busy cycle for the models are found.
We study the load balancing system operating under Join the Shortest Queue (JSQ) in the many-server heavy-traffic regime. If $N$ is the number of servers, we let the difference between the total service rate and the total arrival rate be $N^{1-α}$ with $α>0$. We show that for $α>4$ the average queue length behaves similarly to the classical heavy-traffic regime. Specifically, we prove that the distribution of the average queue length multiplied by $N^{1-α}$ converges to an exponential random variable. Moreover, we show a result analogous to state space collapse. We provide two proofs for our r…
We study a double-ended queue which consists of two classes of customers. Whenever there is a pair of customers from both classes, they are matched and leave the system immediately. The matching follows first-come-first-serve principle. If a customer from one class cannot be matched immediately, he/she will stay in a queue and wait for the upcoming arrivals from the other class. Thus there cannot be non-zero numbers of customers from both classes simultaneously in the system. We also assume that each customer can leave the queue without being matched because of impatience. The arrival processe…
A many-server queueing system is considered in which customers arrive according to a renewal process and have service and patience times that are drawn from two independent sequences of independent, identically distributed random variables. Customers enter service in the order of arrival and are assumed to abandon the queue if the waiting time in queue exceeds the patience time. The state of the system with $N$ servers is represented by a four-component process that consists of the forward recurrence time of the arrival process, a pair of measure-valued processes, one that keeps track of the w…
This paper studies a diffusion model that arises as the limit of a queueing system scheduling problem in the asymptotic heavy traffic regime of Halfin and Whitt. The queueing system consists of several customer classes and many servers working in parallel, grouped in several stations. Servers in different stations offer service to customers of each class at possibly different rates. The control corresponds to selecting what customer class each server serves at each time. The diffusion control problem does not seem to have explicit solutions and therefore a characterization of optimal solutions…
In this paper we study a two-queue polling model with zero switch-over times and $k$-limited service (serve at most $k_i$ customers during one visit period to queue $i$, $i=1,2$) in each queue. The arrival processes at the two queues are Poisson, and the service times are exponentially distributed. By increasing the arrival intensities until one of the queues becomes critically loaded, we derive exact heavy-traffic limits for the joint queue-length distribution using a singular-perturbation technique. It turns out that the number of customers in the stable queue has the same distribution as th…
