We consider a single-server queue with renewal arrivals and i.i.d. service times, in which the server employs either the preemptive Shortest Remaining Processing Time (SRPT) policy, or its non-preemptive variant, Shortest Job First (SJF). We show that for given stochastic primitives (initial condition, arrival and service processes), the model has the same fluid limit under either policy. In particular, we conclude that the well-known queue length optimality of preemptive SRPT is also achieved, asymptotically on fluid scale, by the simpler-to-implement SJF policy. We also conclude that on fluid scale, SJF and SRPT achieve the same performance with respect to response times of the longest-waiting jobs in the system.
We consider Poisson streams of exponentially distributed jobs arriving at each edge of a hypergraph of queues. Upon arrival, an incoming job is rooted to the shortest queue among the corresponding vertices. This generalizes many known models such as power-of-d load balancing and JSQ (join the shortest queue) on generic graphs. We provide a generic condition for stability of this model. We show that some graph topologies lead to a loss of capacity, implying more restrictive stability conditions than in, e.g., complete graphs.
In this paper we propose a highly accurate approximate performance analysis of a heterogeneous server system with a processor sharing service discipline and a general job-size distribution under a generalized join the shortest queue (GJSQ) routing protocol. The GJSQ routing protocol is a natural extension of the well-known join the shortest queue routing policy that takes into account the non-identical service rates in addition to the number of jobs at each server. The performance metrics that are of interest here are the equilibrium distribution and the mean and standard deviation of the numb…
Join the shortest queue (JSQ) refers to networks whose incoming jobs are assigned to the shortest queue from among a randomly chosen subset of the queues in the system. After completion of service at the queue, a job leaves the network. We show that, for all non- idling service disciplines and for general interarrival and service time distributions, such networks are stable when they are subcritical. We then obtain uniform bounds on the tails of the marginal distributions of the equilibria for families of such networks; these bounds are employed to show relative compactness of the marginal dis…
We introduce the \textit{generalized join the shortest queue model with retrials} and two infinite capacity orbit queues. Three independent Poisson streams of jobs, namely a \textit{smart}, and two \textit{dedicated} streams, flow into a single server system, which can hold at most one job. Arriving jobs that find the server occupied are routed to the orbits as follows: Blocked jobs from the \textit{smart} stream are routed to the shortest orbit queue, and in case of a tie, they choose an orbit randomly. Blocked jobs from the \textit{dedicated} streams are routed directly to their orbits. Orbi…
We present a heavy traffic analysis for a single server queue with renewal arrivals and generally distributed i.i.d. service times, in which the server employs the Shortest Remaining Processing Time (SRPT) policy. Under typical heavy traffic assumptions, we prove a diffusion limit theorem for a measure-valued state descriptor, from which we conclude a similar theorem for the queue length process. These results allow us to make some observations on the queue length optimality of SRPT. In particular, they provide the sharpest illustration of the well-known tension between queue length optimality…
This work introduces the join the shortest queue policy in the retrial setting. We consider a Markovian single server retrial system with two infinite capacity orbits. An arriving job finding the server busy, it is forwarded to the least loaded orbit. Otherwise, it is forwarded to an orbit randomly. Orbiting jobs of either type retry to access the server independently. We investigate the stability condition, the stationary tail decay rate, and obtain the equilibrium distribution by using the compensation method.
In this paper, we consider a queueing network with $N$ nodes, each of which has a fixed number $k$ of neighboring nodes, referred to as the $N$ node network with local balancing. We assume that to each of the $N$ nodes, an incoming job (or task) chooses the shortest queue from this node and its neighboring nodes. We construct an appropriate Markov process for this network and find a mean field approximation to this network as $N\rightarrow\infty$, which turns out to be the standard join-the-shortest-queue model.
The fundamental problem in the study of parallel-server systems is that of finding and analyzing `good' routing policies of arriving jobs to the servers. It is well known that, if full information regarding the workload process is available to a central dispatcher, then the {\em join the shortest workload} (JSW) policy, which assigns jobs to the server with the least workload, is the optimal assignment policy, in that it maximizes server utilization, and thus minimizes sojourn times. The {\em join the shortest queue} (JSQ) policy is an efficient dispatching policy when information is available…
We develop a heavy traffic diffusion limit theorem under nonstandard spatial scaling for the queue length process in a single server queue employing shortest remaining processing time (SRPT). For processing time distributions with unbounded support, it has been shown that standard diffusion scaling yields an identically zero limit. We specify an alternative spatial scaling that produces a nonzero limit. Our model allows for renewal arrivals and i.i.d. processing times satisfying a rapid variation condition. We add a corrective spatial scale factor to standard diffusion scaling, and specify con…
We study a single server queue under a processor-sharing type of scheduling policy, where the weights for determining the sharing are given by functions of each job's remaining service(processing) amount, and obtain a fluid limit for the scaled measure-valued system descriptors.
We analyse Join-the-Shortest-Queue in a contemporary scaling regime known as the Non-Degenerate Slowdown regime. Join-the-Shortest-Queue (JSQ) is a classical load balancing policy for queueing systems with multiple parallel servers. Parallel server queueing systems are regularly analysed and dimensioned by diffusion approximations achieved in the Halfin-Whitt scaling regime. However, when jobs must be dispatched to a server upon arrival, we advocate the Non-Degenerate Slowdown regime (NDS) to compare different load-balancing rules. In this paper we identify novel diffusion approximation and ti…
In join the shortest queue networks, incoming jobs are assigned to the shortest queue from among a randomly chosen subset of $D$ queues, in a system of $N$ queues; after completion of service at its queue, a job leaves the network. We also assume that jobs arrive into the system according to a rate-$αN$ Poisson process, $α<1$, with rate-1 service at each queue. When the service at queues is exponentially distributed, it was shown in Vvedenskaya et al. [Probl. Inf. Transm. 32 (1996) 15-29] that the tail of the equilibrium queue size decays doubly exponentially in the limit as $N\rightarrow\inft…
Applications in cloud platforms motivate the study of efficient load balancing under job-server constraints and server heterogeneity. In this paper, we study load balancing on a bipartite graph where left nodes correspond to job types and right nodes correspond to servers, with each edge indicating that a job type can be served by a server. Thus edges represent locality constraints, i.e., each job can only be served at servers which contained certain data and/or machine learning (ML) models. Servers in this system can have heterogeneous service rates. In this setting, we investigate the perfor…
This paper studies load balancing for many-server ($N$ servers) systems. Each server has a buffer of size $b-1,$ and can have at most one job in service and $b-1$ jobs in the buffer. The service time of a job follows the Coxian-2 distribution. We focus on steady-state performance of load balancing policies in the heavy traffic regime such that the normalized load of system is $λ= 1 - N^{-α}$ for $0<α<0.5.$ We identify a set of policies that achieve asymptotic zero waiting. The set of policies include several classical policies such as join-the-shortest-queue (JSQ), join-the-idle-queue (JIQ), i…
This paper considers the steady-state performance of load balancing algorithms in a many-server system with distributed queues. The system has $N$ servers, and each server maintains a local queue with buffer size $b-1,$ i.e. a server can hold at most one job in service and $b-1$ jobs in the queue. Jobs in the same queue are served according to the first-in-first-out (FIFO) order. The system is operated in a heavy-traffic regime such that the workload per server is $λ= 1 - N^{-α}$ for $0.5\leq α<1.$ We identify a set of algorithms such that the steady-state queues have the following universal s…
This paper studies a class of load balancing algorithms for many-server ($N$ servers) systems assuming finite buffer with size $b-1$ (i.e. a server can have at most one job in service and $b-1$ jobs in queue). We focus on steady-state performance of load balancing algorithms in the heavy traffic regime such that the load of system is $λ= 1 - N^{-α}$ for $0<α<0.5,$ which we call sub-Halfin-Whitt regime ($α=0.5$ is the so-called the Halfin-Whitt regime). We establish a sufficient condition under which the probability that an incoming job is routed to an idle server is one asymptotically. The cla…
We consider a system with $N$ parallel servers where incoming jobs are immediately replicated to, say, $d$ servers. Each of the $N$ servers has its own queue and follows a FCFS discipline. As soon as the first job replica is completed, the remaining replicas are abandoned. We investigate the achievable stability region for a quite general workload model with different job types and heterogeneous servers, reflecting job-server affinity relations which may arise from data locality issues and soft compatibility constraints. Under the assumption that job types are known beforehand we show for New-…
The task of completing jobs with decaying value arises in a number of application areas including healthcare operations, communications engineering, and perishable inventory control. We consider a system in which a single server completes a finite sequence of jobs in discrete time while a controller dynamically adjusts the service rate. During service, the value of the job decays so that a greater reward is received for having shorter service times. We incorporate a non-decreasing cost for holding jobs and a non-decreasing cost on the service rate. The controller aims to minimize the total cos…
We consider scheduling to minimize mean response time of the M/G/k queue with unknown job sizes. In the single-server case, the optimal policy is the Gittins policy, but it is not known whether Gittins or any other policy is optimal in the multiserver case. Exactly analyzing the M/G/k under any scheduling policy is intractable, and Gittins is a particularly complicated policy that is hard to analyze even in the single-server case. In this work we introduce monotonic Gittins (M-Gittins), a new variation of the Gittins policy, and show that it minimizes mean response time in the heavy-traffic M/…
In this paper, we study a large system of $N$ servers each with capacity to process at most $C$ simultaneous jobs and an incoming job is routed to a server if it has the lowest occupancy amongst $d$ (out of N) randomly selected servers. A job that is routed to a server with no vacancy is assumed to be blocked and lost. Such randomized policies are referred to JSQ(d) (Join the Shortest Queue out of $d$) policies. Under the assumption that jobs arrive according to a Poisson process with rate $Nλ^{(N)}$ where $λ^{(N)}=σ-\fracβ{\sqrt{N}}$, $σ\in\mb{R}_+$ and $β\in\mb{R}$, we establish functional c…
We consider the c.o.c. redundancy system with $N$ parallel servers where incoming jobs are immediately replicated to $d$ servers chosen uniformly at random (without replacement). A job finishes service as soon as the first replica is completed, after which all the remaining replicas are abandoned. We compare the performance of the first-come first-served (FCFS) and processor-sharing (PS) discipline based on the stability condition, the tail behavior of the latency and the expected latency.
We consider a parallel server system with so-called cancel-on-completion redundancy. There are $n$ servers and multiple job classes $j$. An arriving class $j$ job consists of $d_j$ components, placed on a randomly selected subset of servers; the job service is complete as soon as $k_j$ components out of $d_j$ (with $k_j \le d_j$) complete their service, at which point the unfinished service of all remaining $d_j-k_j$ components is canceled. The system is in general non-work-conserving, in the sense that the average amount of new workload added to the system by an arriving class $j$ job is not …
The shortest-remaining-processing-time (SRPT) scheduling policy has been extensively studied, for more than 50 years, in single-server queues with infinitely patient jobs. Yet, much less is known about its performance in multiserver queues. In this paper, we present the first theoretical analysis of SRPT in multiserver queues with abandonment. In particular, we consider the M/GI/s+GI queue and demonstrate that, in the many-sever overloaded regime, performance in the SRPT queue is equivalent, asymptotically in steady state, to a preemptive two-class priority queue where customers with short ser…
We consider a GI/GI/1 queue with the shortest remaining processing time discipline (SRPT) and light-tailed service times. Our interest is focused on the tail behavior of the sojourn-time distribution. We obtain a general expression for its large-deviations decay rate. The value of this decay rate critically depends on whether there is mass in the endpoint of the service-time distribution or not. An auxiliary priority queue, for which we obtain some new results, plays an important role in our analysis. We apply our SRPT-results to compare SRPT with FIFO from a large-deviations point of view.
For a GI/GI/1 queue, we show that the average sojourn time under the (blind) Randomized Multilevel Feedback algorithm is no worse than that under the Shortest Remaining Processing Time algorithm times a logarithmic function of the system load. Moreover, it is verified that this bound is tight in heavy traffic, up to a constant multiplicative factor. We obtain this result by combining techniques from two disparate areas: competitive analysis and applied probability.
We compute the stationary performance metrics of a single server $M^X/G/1$ queue under a class of generalized processor-sharing scheduling policies that are proposed by Grishechkin. This class of processor-sharing policies allow service capacities to be allocated to jobs based on the amount of service they attained. In [11], Grishechkin derives an integral equation that is satisfied by the Laplace transform of the stationary performance metrics under these policies. Our main focus in this paper is to derive the solution to this integral equation. This is achieved through a series of transforms…
