We study the load balancing system operating under Join the Shortest Queue (JSQ) in the many-server heavy-traffic regime. If $N$ is the number of servers, we let the difference between the total service rate and the total arrival rate be $N^{1-α}$ with $α>0$. We show that for $α>4$ the average queue length behaves similarly to the classical heavy-traffic regime. Specifically, we prove that the distribution of the average queue length multiplied by $N^{1-α}$ converges to an exponential random variable. Moreover, we show a result analogous to state space collapse. We provide two proofs for our result: one using the one-sided Laplace transform, and one using Stein's method. We additionally obtain the rate of convergence in the Wasserstein's distance.
In this paper, we consider a queueing network with $N$ nodes, each of which has a fixed number $k$ of neighboring nodes, referred to as the $N$ node network with local balancing. We assume that to each of the $N$ nodes, an incoming job (or task) chooses the shortest queue from this node and its neighboring nodes. We construct an appropriate Markov process for this network and find a mean field approximation to this network as $N\rightarrow\infty$, which turns out to be the standard join-the-shortest-queue model.
This paper investigates a partially observable queueing system with $N$ nodes in which each node has a dedicated arrival stream. There is an extra arrival stream to balance the load of the system by routing its customers to the shortest queue. In addition, a reward-cost structure is considered to analyze customers' strategic behaviours. The equilibrium and socially optimal strategies are derived for the partially observable mean field limit model. Then, we show that the strategies obtained from the mean field model are good approximations to the model with finite $N$ nodes. Finally, numerical …
We consider queueing systems with n parallel queues under a Join the Shortest Queue (JSQ) policy in the Halfin-Whitt heavy traffic regime. We use the martingale method to prove that a scaled process counting the number of idle servers and queues of length exactly 2 weakly converges to a two-dimensional reflected Ornstein-Uhlenbeck process, while processes counting longer queues converge to a deterministic system decaying to zero in constant time. This limiting system is comparable to that of the traditional Halfin-Whitt model, but there are key differences in the queueing behavior of the JSQ m…
We present a heavy traffic analysis for a single server queue with renewal arrivals and generally distributed i.i.d. service times, in which the server employs the Shortest Remaining Processing Time (SRPT) policy. Under typical heavy traffic assumptions, we prove a diffusion limit theorem for a measure-valued state descriptor, from which we conclude a similar theorem for the queue length process. These results allow us to make some observations on the queue length optimality of SRPT. In particular, they provide the sharpest illustration of the well-known tension between queue length optimality…
Join the shortest queue (JSQ) refers to networks whose incoming jobs are assigned to the shortest queue from among a randomly chosen subset of the queues in the system. After completion of service at the queue, a job leaves the network. We show that, for all non- idling service disciplines and for general interarrival and service time distributions, such networks are stable when they are subcritical. We then obtain uniform bounds on the tails of the marginal distributions of the equilibria for families of such networks; these bounds are employed to show relative compactness of the marginal dis…
We are concerned with an $M/M$-type join the shortest queue ($M/M$-JSQ for short) with $k$ parallel queues for an arbitrary positive integer $k$, where the servers may be heterogeneous. We are interested in the tail asymptotic of the stationary distribution of this queueing model, provided the system is stable. We prove that this asymptotic for the minimum queue length is exactly geometric, and its decay rate is the $k$-th power of the traffic intensity of the corresponding $k$ server queues with a single waiting line. For this, we use two formulations, a quasi-birth-and-death (QBD for short) …
The paper establishes necessary and sufficient conditions for stability of different join-the-shortest-queue models including load-balanced networks with general input and output processes. It is shown, that the necessary and sufficient condition for stability of load-balanced networks is associated with the solution of a linear programming problem precisely formulated in the paper. It is proved, that if the minimum of the objective function of that linear programming problem is less than 1, then the associated load-balanced network is stable.
We consider a join-the-shortest-queue model which is as follows. There are $K$ single FIFO servers and $M$ arrival processes. The customers from a given arrival process can be served only by servers from a certain subset of all servers. The actual destination is the server with the smallest weighted queue length. The arrival processes are assumed to obey a large deviation principle while the service is exponential. A large deviation principle is established for the queue-length process. The action functional is expressed in terms of solutions to mathematical programming problems. The large dev…
We show that the steady-state distribution of the join-the-shortest-queue (JSQ) system converges, in the Halfin-Whitt regime, to its diffusion limit at a rate of at least $1/\sqrt{n}$, where $n$ is the number of servers. Our proof uses Stein's method and, specifically, the recently proposed prelimit generator comparison approach. The JSQ system is non-trivial, high-dimensional, and has a state-space collapse component, and our analysis may serve as a helpful example to readers wishing to apply the approach to their own setting.
In 1987, J.W. Cohen analyzed the so-called Serve the Longest Queue (SLQ) queueing system, where a single server attends two non-symmetric $M/G/1$-type queues, exercising a non-preemptive priority switching policy. Cohen further analyzed in 1998 a non-symmetric 2-queue Markovian system, where newly arriving customers follow the Join the Shortest Queue (JSQ) discipline. The current paper generalizes and extends Cohen's works by studying a combined JSQ-SLQ model, and by broadening the scope of analysis to a non-symmetric 3-queue system, where arriving customers follow the JSQ strategy and a singl…
Customers arrive at rate N times alpha on a network of N single server infinite buffer queues, choose L queues uniformly, join the shortest one, and are served there in turn at rate beta. We let N go to infinity.We prove a functional central limit theorem (CLT) for the tails of the empirical measures of the queue occupations,in a Hilbert space with the weak topology, with limit given by an Ornstein-Uhlenbeck process. The a priori assumption is that the initial data converge.This completes a recent functional CLT in equilibrium result for which convergence for the initial data was not known in …
In this paper we propose a highly accurate approximate performance analysis of a heterogeneous server system with a processor sharing service discipline and a general job-size distribution under a generalized join the shortest queue (GJSQ) routing protocol. The GJSQ routing protocol is a natural extension of the well-known join the shortest queue routing policy that takes into account the non-identical service rates in addition to the number of jobs at each server. The performance metrics that are of interest here are the equilibrium distribution and the mean and standard deviation of the numb…
We consider N single server infinite buffer queues with service rate beta. Customers arrive at rate N times alpha,choose L queues uniformly, and join the shortest one. The stability condition is alpha strictly less than beta. We study in equilibrium the sequence of the fraction of queues of length at least k, in the large N limit. We prove a functional central limit theorem on an infinite-dimensional Hilbert space with its weak topology, with limit a stationary Ornstein-Uhlenbeck process. We use ergodicity and justify the inversion of limits of long times and large sizes N by a compactness-uni…
The scope of this work is twofold: On the one hand, strongly motivated by emerging engineering issues in multiple access communication systems, we investigate the performance of a slotted-time relay-assisted cooperative random access wireless network with collisions and with join the shortest queue relay-routing protocol. For this model, we investigate the stability condition, and apply different methods to derive the joint equilibrium distribution of the queue lengths. On the other hand, using the cooperative communication system as a vehicle for illustration, we investigate and compare three…
We develop a heavy traffic diffusion limit theorem under nonstandard spatial scaling for the queue length process in a single server queue employing shortest remaining processing time (SRPT). For processing time distributions with unbounded support, it has been shown that standard diffusion scaling yields an identically zero limit. We specify an alternative spatial scaling that produces a nonzero limit. Our model allows for renewal arrivals and i.i.d. processing times satisfying a rapid variation condition. We add a corrective spatial scale factor to standard diffusion scaling, and specify con…
We consider Poisson streams of exponentially distributed jobs arriving at each edge of a hypergraph of queues. Upon arrival, an incoming job is rooted to the shortest queue among the corresponding vertices. This generalizes many known models such as power-of-d load balancing and JSQ (join the shortest queue) on generic graphs. We provide a generic condition for stability of this model. We show that some graph topologies lead to a loss of capacity, implying more restrictive stability conditions than in, e.g., complete graphs.
We study a single-server Markovian queueing model with $N$ customer classes in which priority is given to the shortest queue. Under a critical load condition, we establish the diffusion limit of the workload and queue length processes in the form of a Walsh Brownian motion (WBM) living in the union of the $N$ nonnegative coordinate axes in $\mathbb{R}^N$ and a linear transformation thereof. This reveals the following asymptotic behavior. Each time that queues begin to build starting from an empty system, one of them becomes dominant in the sense that it contains nearly all the workload in the …
This paper studies the steady-state properties of the Join the Shortest Queue model in the Halfin-Whitt regime. We focus on the process tracking the number of idle servers, and the number of servers with non-empty buffers. Recently, Eschenfeldt & Gamarnik (2015) proved that a scaled version of this process converges, over finite time intervals, to a two-dimensional diffusion limit as the number of servers goes to infinity. In this paper we prove that the diffusion limit is exponentially ergodic, and that the diffusion scaled sequence of the steady-state number of idle servers and non-empty buf…
This work introduces the join the shortest queue policy in the retrial setting. We consider a Markovian single server retrial system with two infinite capacity orbits. An arriving job finding the server busy, it is forwarded to the least loaded orbit. Otherwise, it is forwarded to an orbit randomly. Orbiting jobs of either type retry to access the server independently. We investigate the stability condition, the stationary tail decay rate, and obtain the equilibrium distribution by using the compensation method.
A simple analytical solution is proposed for the stationary loss system of two parallel queues with finite capacity $K$, in which new customers join the shortest queue, or one of the two with equal probability if their lengths are equal. The arrival process is Poisson, service times at each queue have exponential distributions with the same parameter, and both queues have equal capacity. Using standard generating function arguments, a simple expression for the blocking probability is derived, which as far as we know is original. Using coupling arguments and explicit formulas, comparisons with …
We consider a queueing system consisting of two non-identical exponential servers, where each server has its own dedicated queue and serves the customers in that queue FCFS. Customers arrive according to a Poisson process and join the queue promising the shortest expected delay, which is a natural and near-optimal policy for systems with non-identical servers. This system can be modeled as an inhomogeneous random walk in the quadrant. By stretching the boundaries of the compensation approach we prove that the equilibrium distribution of this random walk can be expressed as a series of product-…
We consider a single-server queue with renewal arrivals and i.i.d. service times, in which the server employs either the preemptive Shortest Remaining Processing Time (SRPT) policy, or its non-preemptive variant, Shortest Job First (SJF). We show that for given stochastic primitives (initial condition, arrival and service processes), the model has the same fluid limit under either policy. In particular, we conclude that the well-known queue length optimality of preemptive SRPT is also achieved, asymptotically on fluid scale, by the simpler-to-implement SJF policy. We also conclude that on flui…
We consider N single server infinite buffer queues with service rate β. Customers arrive at rate Nα, choose L queues uniformly, and join the shortest. We study the processes R^N for large N, where R^N_t(k) is the fraction of queues of length at least k at time t. Laws of large numbers (LLNs) are known. We consider certain Hilbert spaces with the weak topology. First, we prove a functional central limit theorem (CLT) under the a priori assumption that the initial data R^N_0 satisfy the corresponding CLT. We use a compactness-uniqueness method, and the limit is characterized as an Ornstein-Uhlen…
We introduce the \textit{generalized join the shortest queue model with retrials} and two infinite capacity orbit queues. Three independent Poisson streams of jobs, namely a \textit{smart}, and two \textit{dedicated} streams, flow into a single server system, which can hold at most one job. Arriving jobs that find the server occupied are routed to the orbits as follows: Blocked jobs from the \textit{smart} stream are routed to the shortest orbit queue, and in case of a tie, they choose an orbit randomly. Blocked jobs from the \textit{dedicated} streams are routed directly to their orbits. Orbi…
Consider a system of $N$ parallel single-server queues with unit-exponential service time distribution and a single dispatcher where tasks arrive as a Poisson process of rate $λ(N)$. When a task arrives, the dispatcher assigns it to one of the servers according to the Join-the-Shortest Queue (JSQ) policy. Eschenfeldt and Gamarnik (2015) established that in the Halfin-Whitt regime where $(N-λ(N))/\sqrt{N}\toβ>0$ as $N\to\infty$, appropriately scaled occupancy measure of the system under the JSQ policy converges weakly on any finite time interval to a certain diffusion process as $N\to\infty$. R…
Consider a system of $N$ parallel single-server queues with unit-exponential service time distribution and a single dispatcher where tasks arrive as a Poisson process of rate $λ(N)$. When a task arrives, the dispatcher assigns it to one of the servers according to the Join-the-Shortest Queue (JSQ) policy. Eschenfeldt and Gamarnik (Math. Oper. Res., 43(3):867-886, 2018) identified a novel limiting diffusion process that arises as the weak-limit of the appropriately scaled occupancy measure of the system under the JSQ policy in the Halfin-Whitt regime, where $(N - λ(N)) / \sqrt{N} \to β> 0$ as $…
We consider a system of $N$ parallel queues with identical exponential service rates and a single dispatcher where tasks arrive as a Poisson process. When a task arrives, the dispatcher always assigns it to an idle server, if there is any, and to a server with the shortest queue among $d$ randomly selected servers otherwise $(1 \leq d \leq N)$. This load balancing scheme subsumes the so-called Join-the-Idle Queue (JIQ) policy $(d = 1)$ and the celebrated Join-the-Shortest Queue (JSQ) policy $(d = N)$ as two crucial special cases. We develop a stochastic coupling construction to obtain the diff…
In join the shortest queue networks, incoming jobs are assigned to the shortest queue from among a randomly chosen subset of $D$ queues, in a system of $N$ queues; after completion of service at its queue, a job leaves the network. We also assume that jobs arrive into the system according to a rate-$αN$ Poisson process, $α<1$, with rate-1 service at each queue. When the service at queues is exponentially distributed, it was shown in Vvedenskaya et al. [Probl. Inf. Transm. 32 (1996) 15-29] that the tail of the equilibrium queue size decays doubly exponentially in the limit as $N\rightarrow\inft…
This paper considers a work-conserving FIFO single-server queue with multiple batch Markovian arrival streams governed by a continuous-time finite-state Markov chain. A particular feature of this queue is that service time distributions of customers may be different for different arrival streams. After briefly discussing the actual waiting time distributions of customers from respective arrival streams, we derive a formula for the vector generating function of the time-average joint queue length distribution in terms of the virtual waiting time distribution. Further assuming the discrete phase…
We introduce the appropriate framework for pathwise comparison of multiple server queues under general stationary ergodic assumptions. We show in what sense it is better to have more servers for a system under FCFS ('First Come, First Served') or equivalently, more queues in a system of parallel queues under the JSW ('Join the Shortest Workload') allocation policy. This comparison result is based on the recursive representation of Kiefer and Wolfowitz, and on a non-mass conservative generalization of the Schur-Convex semi-ordering. We also show that the latter result does not hold true in gene…
We analyse Join-the-Shortest-Queue in a contemporary scaling regime known as the Non-Degenerate Slowdown regime. Join-the-Shortest-Queue (JSQ) is a classical load balancing policy for queueing systems with multiple parallel servers. Parallel server queueing systems are regularly analysed and dimensioned by diffusion approximations achieved in the Halfin-Whitt scaling regime. However, when jobs must be dispatched to a server upon arrival, we advocate the Non-Degenerate Slowdown regime (NDS) to compare different load-balancing rules. In this paper we identify novel diffusion approximation and ti…
The shortest-remaining-processing-time (SRPT) scheduling policy has been extensively studied, for more than 50 years, in single-server queues with infinitely patient jobs. Yet, much less is known about its performance in multiserver queues. In this paper, we present the first theoretical analysis of SRPT in multiserver queues with abandonment. In particular, we consider the M/GI/s+GI queue and demonstrate that, in the many-sever overloaded regime, performance in the SRPT queue is equivalent, asymptotically in steady state, to a preemptive two-class priority queue where customers with short ser…
In this paper we analyze an $M/M/1$ queueing system with an arbitrary number of customer classes, with class-dependent exponential service rates and preemptive priorities between classes. The queuing system can be described by a multi-dimensional Markov process, where the coordinates keep track of the number of customers of each class in the system. Based on matrix-analytic techniques and probabilistic arguments we develop a recursive method for the exact determination of the equilibrium joint queue length distribution. The method is applied to a spare parts logistics problem to illustrate the…
This paper studies the heavy-traffic asymptotics for the multiclass FIFO M${}^X$/G/1 queue. We first derive the probability generating function of the joint queue length distribution. Using the probability generating function, we then present heavy-traffic asymptotic formulas for the joint queue length distribution and its joint moments (i.e., the joint queue length moments). These formulas are proved under weaker conditions on the service time distributions, compared to the ones reported in the literature. This fact leads us to conjectures that some of the conditions made in the literature ar…
We investigate, under general stationary ergodic assumptions, the stability of systems of $S$ parallel queues in which any incoming customer joins the queue of the server having the $p+1$-th shortest workload ($p < S$), or a free server if any. This change in the allocation policy makes the analysis much more challenging with respect to the classical FCFS model with $S$ servers, as it leads to the non-monotonicity of the underlying stochastic recursion. We provide sufficient conditions of existence of a stationary workload, which indicate a "splitting" of the system in heavy traffic, into a lo…
We consider a model for transitory queues in which only a finite number of customers can join. The queue thus operates over a finite time horizon. In this system, also known as the $Δ_{(i)}/G/1$ queue, the customers decide independently when to join the queue by sampling their arrival time from a common distribution. We prove that, when the queue satisfies a certain heavy-traffic condition and under the additional assumption that the second moment of the service time is finite, the rescaled queue length process converges to a reflected Brownian motion with parabolic drift. Our result holds for…
A parallel server system with $n$ identical servers is considered. The service time distribution has a finite mean $1/μ$, but otherwise is arbitrary. Arriving customers are be routed to one of the servers immediately upon arrival. Join-Idle-Queue routing algorithm is studied, under which an arriving customer is sent to an idle server, if such is available, and to a randomly uniformly chosen server, otherwise. We consider the asymptotic regime where $n\to\infty$ and the customer input flow rate is $λn$. Under the condition $λ/μ<1/2$, we prove that, as $n\to\infty$, the sequence of (appropriatel…
We consider a queueing system with $n$ parallel queues operating according to the so-called "supermarket model" in which arriving customers join the shortest of $d$ randomly selected queues. Assuming rate $nλ_{n}$ Poisson arrivals and rate $1$ exponentially distributed service times, we consider this model in the heavy traffic regime, described by $λ_{n}\uparrow 1$ as $n\to\infty$. We give a simple expectation argument establishing that majority of queues have steady state length at least $\log_d(1-λ_{n})^{-1} - O(1)$ with probability approaching one as $n\rightarrow\infty$, implying the same …
In this paper, we study an $N$ server fork-join queue with nearly deterministic arrival and service times. Specifically, we present a fluid limit for the maximum queue length as $N\to\infty$. This fluid limit depends on the initial number of tasks. In order to prove these results, we develop extreme value theory and diffusion approximations for the queue lengths.
In this paper, we study the tail behavior of $\max_{i\leq N}\sup_{s>0}\left(W_i(s)+W_A(s)-βs\right)$ as $N\to\infty$, with $(W_i,i\leq N)$ i.i.d. Brownian motions and $W_A$ an independent Brownian motion. This random variable can be seen as the maximum of $N$ mutually dependent Brownian queues, which in turn can be interpreted as the backlog in a Brownian fork-join queue. In previous work, we have shown that this random variable centers around $\frac{σ^2}{2β}\log N$. Here, we analyze the rare-event that this random variable reaches the value $(\frac{σ^2}{2β}+a)\log N$, with $a>0$. It turns out…
This paper considers the steady-state performance of load balancing algorithms in a many-server system with distributed queues. The system has $N$ servers, and each server maintains a local queue with buffer size $b-1,$ i.e. a server can hold at most one job in service and $b-1$ jobs in the queue. Jobs in the same queue are served according to the first-in-first-out (FIFO) order. The system is operated in a heavy-traffic regime such that the workload per server is $λ= 1 - N^{-α}$ for $0.5\leq α<1.$ We identify a set of algorithms such that the steady-state queues have the following universal s…
In the supermarket model there are n queues, each with a unit rate server. Customers arrive in a Poisson process at rate λn, where 0<λ<1. Each customer chooses d > 2 queues uniformly at random, and joins a shortest one. It is known that the equilibrium distribution of a typical queue length converges to a certain explicit limiting distribution as n -> oo. We quantify the rate of convergence by showing that the total variation distance between the equilibrium distribution and the limiting distribution is essentially of order n^{-1}; and we give a corresponding result for systems starting from q…
We consider a single server queue that serves a finite population of $n$ customers that will enter the queue (require service) only once, also known as the $Δ_{(i)}/G/1$ queue. This paper presents a method for analyzing heavy-traffic behavior by using uniform acceleration, which simultaneously lets $n$ and the service rate grow large, while the initial resource utilization approaches one. A key feature of the model is that, as time progresses, more customers have joined the queue, and fewer customers can potentially join. This diminishing population gives rise to a class of reflected stochasti…
