The paper establishes necessary and sufficient conditions for stability of different join-the-shortest-queue models including load-balanced networks with general input and output processes. It is shown, that the necessary and sufficient condition for stability of load-balanced networks is associated with the solution of a linear programming problem precisely formulated in the paper. It is proved, that if the minimum of the objective function of that linear programming problem is less than 1, then the associated load-balanced network is stable.
In this paper, we consider a queueing network with $N$ nodes, each of which has a fixed number $k$ of neighboring nodes, referred to as the $N$ node network with local balancing. We assume that to each of the $N$ nodes, an incoming job (or task) chooses the shortest queue from this node and its neighboring nodes. We construct an appropriate Markov process for this network and find a mean field approximation to this network as $N\rightarrow\infty$, which turns out to be the standard join-the-shortest-queue model.
This paper investigates a partially observable queueing system with $N$ nodes in which each node has a dedicated arrival stream. There is an extra arrival stream to balance the load of the system by routing its customers to the shortest queue. In addition, a reward-cost structure is considered to analyze customers' strategic behaviours. The equilibrium and socially optimal strategies are derived for the partially observable mean field limit model. Then, we show that the strategies obtained from the mean field model are good approximations to the model with finite $N$ nodes. Finally, numerical …
We consider queueing systems with n parallel queues under a Join the Shortest Queue (JSQ) policy in the Halfin-Whitt heavy traffic regime. We use the martingale method to prove that a scaled process counting the number of idle servers and queues of length exactly 2 weakly converges to a two-dimensional reflected Ornstein-Uhlenbeck process, while processes counting longer queues converge to a deterministic system decaying to zero in constant time. This limiting system is comparable to that of the traditional Halfin-Whitt model, but there are key differences in the queueing behavior of the JSQ m…
We present a heavy traffic analysis for a single server queue with renewal arrivals and generally distributed i.i.d. service times, in which the server employs the Shortest Remaining Processing Time (SRPT) policy. Under typical heavy traffic assumptions, we prove a diffusion limit theorem for a measure-valued state descriptor, from which we conclude a similar theorem for the queue length process. These results allow us to make some observations on the queue length optimality of SRPT. In particular, they provide the sharpest illustration of the well-known tension between queue length optimality…
Join the shortest queue (JSQ) refers to networks whose incoming jobs are assigned to the shortest queue from among a randomly chosen subset of the queues in the system. After completion of service at the queue, a job leaves the network. We show that, for all non- idling service disciplines and for general interarrival and service time distributions, such networks are stable when they are subcritical. We then obtain uniform bounds on the tails of the marginal distributions of the equilibria for families of such networks; these bounds are employed to show relative compactness of the marginal dis…
We are concerned with an $M/M$-type join the shortest queue ($M/M$-JSQ for short) with $k$ parallel queues for an arbitrary positive integer $k$, where the servers may be heterogeneous. We are interested in the tail asymptotic of the stationary distribution of this queueing model, provided the system is stable. We prove that this asymptotic for the minimum queue length is exactly geometric, and its decay rate is the $k$-th power of the traffic intensity of the corresponding $k$ server queues with a single waiting line. For this, we use two formulations, a quasi-birth-and-death (QBD for short) …
We consider a join-the-shortest-queue model which is as follows. There are $K$ single FIFO servers and $M$ arrival processes. The customers from a given arrival process can be served only by servers from a certain subset of all servers. The actual destination is the server with the smallest weighted queue length. The arrival processes are assumed to obey a large deviation principle while the service is exponential. A large deviation principle is established for the queue-length process. The action functional is expressed in terms of solutions to mathematical programming problems. The large dev…
We show that the steady-state distribution of the join-the-shortest-queue (JSQ) system converges, in the Halfin-Whitt regime, to its diffusion limit at a rate of at least $1/\sqrt{n}$, where $n$ is the number of servers. Our proof uses Stein's method and, specifically, the recently proposed prelimit generator comparison approach. The JSQ system is non-trivial, high-dimensional, and has a state-space collapse component, and our analysis may serve as a helpful example to readers wishing to apply the approach to their own setting.
We study the load balancing system operating under Join the Shortest Queue (JSQ) in the many-server heavy-traffic regime. If $N$ is the number of servers, we let the difference between the total service rate and the total arrival rate be $N^{1-α}$ with $α>0$. We show that for $α>4$ the average queue length behaves similarly to the classical heavy-traffic regime. Specifically, we prove that the distribution of the average queue length multiplied by $N^{1-α}$ converges to an exponential random variable. Moreover, we show a result analogous to state space collapse. We provide two proofs for our r…
In 1987, J.W. Cohen analyzed the so-called Serve the Longest Queue (SLQ) queueing system, where a single server attends two non-symmetric $M/G/1$-type queues, exercising a non-preemptive priority switching policy. Cohen further analyzed in 1998 a non-symmetric 2-queue Markovian system, where newly arriving customers follow the Join the Shortest Queue (JSQ) discipline. The current paper generalizes and extends Cohen's works by studying a combined JSQ-SLQ model, and by broadening the scope of analysis to a non-symmetric 3-queue system, where arriving customers follow the JSQ strategy and a singl…
Customers arrive at rate N times alpha on a network of N single server infinite buffer queues, choose L queues uniformly, join the shortest one, and are served there in turn at rate beta. We let N go to infinity.We prove a functional central limit theorem (CLT) for the tails of the empirical measures of the queue occupations,in a Hilbert space with the weak topology, with limit given by an Ornstein-Uhlenbeck process. The a priori assumption is that the initial data converge.This completes a recent functional CLT in equilibrium result for which convergence for the initial data was not known in …
In this paper we propose a highly accurate approximate performance analysis of a heterogeneous server system with a processor sharing service discipline and a general job-size distribution under a generalized join the shortest queue (GJSQ) routing protocol. The GJSQ routing protocol is a natural extension of the well-known join the shortest queue routing policy that takes into account the non-identical service rates in addition to the number of jobs at each server. The performance metrics that are of interest here are the equilibrium distribution and the mean and standard deviation of the numb…
We consider N single server infinite buffer queues with service rate beta. Customers arrive at rate N times alpha,choose L queues uniformly, and join the shortest one. The stability condition is alpha strictly less than beta. We study in equilibrium the sequence of the fraction of queues of length at least k, in the large N limit. We prove a functional central limit theorem on an infinite-dimensional Hilbert space with its weak topology, with limit a stationary Ornstein-Uhlenbeck process. We use ergodicity and justify the inversion of limits of long times and large sizes N by a compactness-uni…
The scope of this work is twofold: On the one hand, strongly motivated by emerging engineering issues in multiple access communication systems, we investigate the performance of a slotted-time relay-assisted cooperative random access wireless network with collisions and with join the shortest queue relay-routing protocol. For this model, we investigate the stability condition, and apply different methods to derive the joint equilibrium distribution of the queue lengths. On the other hand, using the cooperative communication system as a vehicle for illustration, we investigate and compare three…
We develop a heavy traffic diffusion limit theorem under nonstandard spatial scaling for the queue length process in a single server queue employing shortest remaining processing time (SRPT). For processing time distributions with unbounded support, it has been shown that standard diffusion scaling yields an identically zero limit. We specify an alternative spatial scaling that produces a nonzero limit. Our model allows for renewal arrivals and i.i.d. processing times satisfying a rapid variation condition. We add a corrective spatial scale factor to standard diffusion scaling, and specify con…
We consider Poisson streams of exponentially distributed jobs arriving at each edge of a hypergraph of queues. Upon arrival, an incoming job is rooted to the shortest queue among the corresponding vertices. This generalizes many known models such as power-of-d load balancing and JSQ (join the shortest queue) on generic graphs. We provide a generic condition for stability of this model. We show that some graph topologies lead to a loss of capacity, implying more restrictive stability conditions than in, e.g., complete graphs.
We study a single-server Markovian queueing model with $N$ customer classes in which priority is given to the shortest queue. Under a critical load condition, we establish the diffusion limit of the workload and queue length processes in the form of a Walsh Brownian motion (WBM) living in the union of the $N$ nonnegative coordinate axes in $\mathbb{R}^N$ and a linear transformation thereof. This reveals the following asymptotic behavior. Each time that queues begin to build starting from an empty system, one of them becomes dominant in the sense that it contains nearly all the workload in the …
This paper studies the steady-state properties of the Join the Shortest Queue model in the Halfin-Whitt regime. We focus on the process tracking the number of idle servers, and the number of servers with non-empty buffers. Recently, Eschenfeldt & Gamarnik (2015) proved that a scaled version of this process converges, over finite time intervals, to a two-dimensional diffusion limit as the number of servers goes to infinity. In this paper we prove that the diffusion limit is exponentially ergodic, and that the diffusion scaled sequence of the steady-state number of idle servers and non-empty buf…
This work introduces the join the shortest queue policy in the retrial setting. We consider a Markovian single server retrial system with two infinite capacity orbits. An arriving job finding the server busy, it is forwarded to the least loaded orbit. Otherwise, it is forwarded to an orbit randomly. Orbiting jobs of either type retry to access the server independently. We investigate the stability condition, the stationary tail decay rate, and obtain the equilibrium distribution by using the compensation method.
A simple analytical solution is proposed for the stationary loss system of two parallel queues with finite capacity $K$, in which new customers join the shortest queue, or one of the two with equal probability if their lengths are equal. The arrival process is Poisson, service times at each queue have exponential distributions with the same parameter, and both queues have equal capacity. Using standard generating function arguments, a simple expression for the blocking probability is derived, which as far as we know is original. Using coupling arguments and explicit formulas, comparisons with …
We consider a queueing system consisting of two non-identical exponential servers, where each server has its own dedicated queue and serves the customers in that queue FCFS. Customers arrive according to a Poisson process and join the queue promising the shortest expected delay, which is a natural and near-optimal policy for systems with non-identical servers. This system can be modeled as an inhomogeneous random walk in the quadrant. By stretching the boundaries of the compensation approach we prove that the equilibrium distribution of this random walk can be expressed as a series of product-…
We consider a single-server queue with renewal arrivals and i.i.d. service times, in which the server employs either the preemptive Shortest Remaining Processing Time (SRPT) policy, or its non-preemptive variant, Shortest Job First (SJF). We show that for given stochastic primitives (initial condition, arrival and service processes), the model has the same fluid limit under either policy. In particular, we conclude that the well-known queue length optimality of preemptive SRPT is also achieved, asymptotically on fluid scale, by the simpler-to-implement SJF policy. We also conclude that on flui…
We consider N single server infinite buffer queues with service rate β. Customers arrive at rate Nα, choose L queues uniformly, and join the shortest. We study the processes R^N for large N, where R^N_t(k) is the fraction of queues of length at least k at time t. Laws of large numbers (LLNs) are known. We consider certain Hilbert spaces with the weak topology. First, we prove a functional central limit theorem (CLT) under the a priori assumption that the initial data R^N_0 satisfy the corresponding CLT. We use a compactness-uniqueness method, and the limit is characterized as an Ornstein-Uhlen…
We introduce the \textit{generalized join the shortest queue model with retrials} and two infinite capacity orbit queues. Three independent Poisson streams of jobs, namely a \textit{smart}, and two \textit{dedicated} streams, flow into a single server system, which can hold at most one job. Arriving jobs that find the server occupied are routed to the orbits as follows: Blocked jobs from the \textit{smart} stream are routed to the shortest orbit queue, and in case of a tie, they choose an orbit randomly. Blocked jobs from the \textit{dedicated} streams are routed directly to their orbits. Orbi…
Consider a system of $N$ parallel single-server queues with unit-exponential service time distribution and a single dispatcher where tasks arrive as a Poisson process of rate $λ(N)$. When a task arrives, the dispatcher assigns it to one of the servers according to the Join-the-Shortest Queue (JSQ) policy. Eschenfeldt and Gamarnik (2015) established that in the Halfin-Whitt regime where $(N-λ(N))/\sqrt{N}\toβ>0$ as $N\to\infty$, appropriately scaled occupancy measure of the system under the JSQ policy converges weakly on any finite time interval to a certain diffusion process as $N\to\infty$. R…
Consider a system of $N$ parallel single-server queues with unit-exponential service time distribution and a single dispatcher where tasks arrive as a Poisson process of rate $λ(N)$. When a task arrives, the dispatcher assigns it to one of the servers according to the Join-the-Shortest Queue (JSQ) policy. Eschenfeldt and Gamarnik (Math. Oper. Res., 43(3):867-886, 2018) identified a novel limiting diffusion process that arises as the weak-limit of the appropriately scaled occupancy measure of the system under the JSQ policy in the Halfin-Whitt regime, where $(N - λ(N)) / \sqrt{N} \to β> 0$ as $…
We consider a system of $N$ parallel queues with identical exponential service rates and a single dispatcher where tasks arrive as a Poisson process. When a task arrives, the dispatcher always assigns it to an idle server, if there is any, and to a server with the shortest queue among $d$ randomly selected servers otherwise $(1 \leq d \leq N)$. This load balancing scheme subsumes the so-called Join-the-Idle Queue (JIQ) policy $(d = 1)$ and the celebrated Join-the-Shortest Queue (JSQ) policy $(d = N)$ as two crucial special cases. We develop a stochastic coupling construction to obtain the diff…
In join the shortest queue networks, incoming jobs are assigned to the shortest queue from among a randomly chosen subset of $D$ queues, in a system of $N$ queues; after completion of service at its queue, a job leaves the network. We also assume that jobs arrive into the system according to a rate-$αN$ Poisson process, $α<1$, with rate-1 service at each queue. When the service at queues is exponentially distributed, it was shown in Vvedenskaya et al. [Probl. Inf. Transm. 32 (1996) 15-29] that the tail of the equilibrium queue size decays doubly exponentially in the limit as $N\rightarrow\inft…
We analyse Join-the-Shortest-Queue in a contemporary scaling regime known as the Non-Degenerate Slowdown regime. Join-the-Shortest-Queue (JSQ) is a classical load balancing policy for queueing systems with multiple parallel servers. Parallel server queueing systems are regularly analysed and dimensioned by diffusion approximations achieved in the Halfin-Whitt scaling regime. However, when jobs must be dispatched to a server upon arrival, we advocate the Non-Degenerate Slowdown regime (NDS) to compare different load-balancing rules. In this paper we identify novel diffusion approximation and ti…
We investigate, under general stationary ergodic assumptions, the stability of systems of $S$ parallel queues in which any incoming customer joins the queue of the server having the $p+1$-th shortest workload ($p < S$), or a free server if any. This change in the allocation policy makes the analysis much more challenging with respect to the classical FCFS model with $S$ servers, as it leads to the non-monotonicity of the underlying stochastic recursion. We provide sufficient conditions of existence of a stationary workload, which indicate a "splitting" of the system in heavy traffic, into a lo…
We consider a model for transitory queues in which only a finite number of customers can join. The queue thus operates over a finite time horizon. In this system, also known as the $Δ_{(i)}/G/1$ queue, the customers decide independently when to join the queue by sampling their arrival time from a common distribution. We prove that, when the queue satisfies a certain heavy-traffic condition and under the additional assumption that the second moment of the service time is finite, the rescaled queue length process converges to a reflected Brownian motion with parabolic drift. Our result holds for…
A parallel server system with $n$ identical servers is considered. The service time distribution has a finite mean $1/μ$, but otherwise is arbitrary. Arriving customers are be routed to one of the servers immediately upon arrival. Join-Idle-Queue routing algorithm is studied, under which an arriving customer is sent to an idle server, if such is available, and to a randomly uniformly chosen server, otherwise. We consider the asymptotic regime where $n\to\infty$ and the customer input flow rate is $λn$. Under the condition $λ/μ<1/2$, we prove that, as $n\to\infty$, the sequence of (appropriatel…
In this paper, we study an $N$ server fork-join queue with nearly deterministic arrival and service times. Specifically, we present a fluid limit for the maximum queue length as $N\to\infty$. This fluid limit depends on the initial number of tasks. In order to prove these results, we develop extreme value theory and diffusion approximations for the queue lengths.
In this paper, we study the tail behavior of $\max_{i\leq N}\sup_{s>0}\left(W_i(s)+W_A(s)-βs\right)$ as $N\to\infty$, with $(W_i,i\leq N)$ i.i.d. Brownian motions and $W_A$ an independent Brownian motion. This random variable can be seen as the maximum of $N$ mutually dependent Brownian queues, which in turn can be interpreted as the backlog in a Brownian fork-join queue. In previous work, we have shown that this random variable centers around $\frac{σ^2}{2β}\log N$. Here, we analyze the rare-event that this random variable reaches the value $(\frac{σ^2}{2β}+a)\log N$, with $a>0$. It turns out…
This paper considers the steady-state performance of load balancing algorithms in a many-server system with distributed queues. The system has $N$ servers, and each server maintains a local queue with buffer size $b-1,$ i.e. a server can hold at most one job in service and $b-1$ jobs in the queue. Jobs in the same queue are served according to the first-in-first-out (FIFO) order. The system is operated in a heavy-traffic regime such that the workload per server is $λ= 1 - N^{-α}$ for $0.5\leq α<1.$ We identify a set of algorithms such that the steady-state queues have the following universal s…
We consider a system of $N$ parallel single-server queues with unit exponential service rates and a single dispatcher where tasks arrive as a Poisson process of rate $λ(N)$. When a task arrives, the dispatcher assigns it to a server with the shortest queue among $d(N)$ randomly selected servers ($1 \leq d(N) \leq N$). This load balancing strategy is referred to as a JSQ($d(N)$) scheme, marking that it subsumes the celebrated Join-the-Shortest Queue (JSQ) policy as a crucial special case for $d(N) = N$. We construct a stochastic coupling to bound the difference in the queue length processes bet…
In this paper, we study the maximum waiting time $\max_{i\leq N}W_i(\cdot)$ in an $N$-server fork-join queue with heavy-tailed services as $N\to\infty$. The service times are the product of two random variables. One random variable has a regularly varying tail probability and is the same among all $N$ servers, and one random variable is Weibull distributed and is independent and identically distributed among all servers. This setup has the physical interpretation that if a job has a large size, then all the subtasks have large sizes, with some variability described by the Weibull-distributed p…
This paper considers a population process on a dynamically evolving graph, which can be alternatively interpreted as a queueing network. The queues are of infinite-server type, entailing that at each node all customers present are served in parallel. The links that connect the queues have the special feature that they are unreliable, in the sense that their status alternates between 'up' and 'down'. If a link between two nodes is down, with a fixed probability each of the clients attempting to use that link is lost; otherwise the client remains at the origin node and reattempts using the link …
Applications in cloud platforms motivate the study of efficient load balancing under job-server constraints and server heterogeneity. In this paper, we study load balancing on a bipartite graph where left nodes correspond to job types and right nodes correspond to servers, with each edge indicating that a job type can be served by a server. Thus edges represent locality constraints, i.e., each job can only be served at servers which contained certain data and/or machine learning (ML) models. Servers in this system can have heterogeneous service rates. In this setting, we investigate the perfor…
We consider a pair of coupled queues driven by independent spectrally-positive Levy processes. With respect to the bi-variate workload process this framework includes both the coupled processor model and the two-server fluid network with independent Levy inputs. We identify the joint transform of the stationary workload distribution in terms of Wiener-Hopf factors corresponding to two auxiliary Levy processes with explicit Laplace exponents. We reinterpret and extend the ideas of Cohen and Boxma (1983) to provide a general and uniform result with a neat transform expression.
This paper considers a network of infinite-server queues with the special feature that, triggered by specific events, the network population vector may undergo a linear transformation (a `multiplicative transition'). For this model we characterize the joint probability generating function in terms of a system of partial differential equations; this system enables the evaluation of (transient as well as stationary) moments. We show that several relevant systems fit in the framework developed, such as networks of retrial queues, networks in which jobs can be rerouted when links fail, and storage…
This paper extends previous work by the authors. We consider the local time process of a strong Markov process, add negative drift, and reflect it à la Skorokhod. The resulting process is used to model a fluid queue. We derive an expression for the joint law of the duration of an excursion, the maximum value of the process on it, and the time distance between successive excursions. We work with a properly constructed stationary version of the process. Examples are also given in the paper.
Randomized load balancing networks arise in a variety of applications, and allow for efficient sharing of resources, while being relatively easy to implement. We consider a network of parallel queues in which incoming jobs with independent and identically distributed service times are assigned to the shortest queue among a randomly chosen subset of $d$ queues, and leave the network on completion of service. Prior work on dynamical properties of this model has focused on the case of exponential service distributions. In this work, we analyze the more realistic case of general service distributi…
The basic load balancing scenario involves a single dispatcher where tasks arrive that must immediately be forwarded to one of $N$ single-server queues. We discuss recent advances on scalable load balancing schemes which provide favorable delay performance when $N$ grows large, and yet only require minimal implementation overhead. Join-the-Shortest-Queue (JSQ) yields vanishing delays as $N$ grows large, as in a centralized queueing arrangement, but involves a prohibitive communication burden. In contrast, power-of-$d$ or JSQ($d$) schemes that assign an incoming task to a server with the shorte…
