The standard setting for studying parallel server systems (PSS) at the diffusion scale is based on the heavy traffic condition (HTC), which assumes that the underlying static allocation linear program (LP) is critical and has a unique solution. This solution determines the graph of basic activities, which identifies the set of activities (i.e., class-server pairs) that are operational. In this paper we explore the extended HTC, where the LP is merely assumed to be critical. Because multiple solutions are allowed, multiple sets of operational activities, referred to as modes, are available. Formally, the scaling limit for the control problem associated with the model is given by a so called workload control problem (WCP) in which a cost associated with a diffusion process is to be minimized by dynamically switching between these modes. Our main result is that the WCP's value constitutes an asymptotic lower bound on the cost associated with the PSS model.
This paper studies a 2-class, 2-server parallel server system under the recently introduced extended heavy traffic condition, which states that the underlying 'static allocation' linear program (LP) is critical, but does not require that it has a unique solution. The main result is the construction of policies that asymptotically achieve a lower bound, proved in [1], on an expected discounted linear combination of diffusion-scaled queue lengths, and are therefore asymptotically optimal (AO). Each extreme point solution to the LP determines a control mode, i.e., a set of activities (class--serv…
We consider a large queueing system that consists of many strategic servers that are weakly interacting. Each server processes jobs from its unique critically loaded buffer and controls the rate of arrivals and departures associated with its queue to minimize its expected cost. The rates and the cost functions in addition to depending on the control action, can depend, in a symmetric fashion, on the size of the individual queue and the empirical measure of the states of all queues in the system. In order to determine an approximate Nash equilibrium for this finite player game we construct a La…
We consider a queue with renewal arrivals and n exponential servers in the Halfin-Whitt heavy traffic regime, where n and the arrival rate increase without bound, so that a critical loading condition holds. Server k serves at rate $μ_k $, and the empirical distribution of the $μ_k $ is assumed to converge weakly. We show that very little information on the service rates is required for a routing mechanism to perform well. More precisely, we construct a routing mechanism that has access to a single sample from the service time distribution of each of $n$ to the power of $1/2 + ε$ randomly selec…
We study a stochastic network that consists of a set of servers processing multiple classes of jobs. Each class of jobs requires a concurrent occupancy of several servers while being processed, and each server is shared among the job classes in a head-of-the-line processor-sharing mechanism. The allocation of the service capacities is a real-time control mechanism: in each network state, the control is the solution to an optimization problem that maximizes a general utility function. Whereas this resource control optimizes in a ``greedy'' fashion, with respect to each state, we establish its a…
A G/M/N queue is considered in the moderate deviation heavy traffic regime. The rate function for the customers-in-system process is obtained for the single class model. A risk-sensitive type control problem is considered for multi-class G/M/N model under the moderate deviation scaling and shown that the optimal control problem is related to a differential game problem.
Communication is a crucial phase in the context of distributed training. Because parameter server (PS) frequently experiences network congestion, recent studies have found that training paradigms without a centralized server outperform the traditional server-based paradigms in terms of communication efficiency. However, with the increasing growth of model sizes, these server-free paradigms are also confronted with substantial communication overhead that seriously deteriorates the performance of distributed training. In this paper, we focus on communication efficiency of two serverless paradigm…
We consider learning-based variants of the $c μ$ rule for scheduling in single and parallel server settings of multi-class queueing systems. In the single server setting, the $c μ$ rule is known to minimize the expected holding-cost (weighted queue-lengths summed over classes and a fixed time horizon). We focus on the problem where the service rates $μ$ are unknown with the holding-cost regret (regret against the $c μ$ rule with known $μ$) as our objective. We show that the greedy algorithm that uses empirically learned service rates results in a constant holding-cost regret (the regret is ind…
This paper considers a Markov decision model for profit maximization of a cloud computing service provider catering to customers submitting jobs with firm real-time random deadlines. Customers are charged on a per-job basis, receiving a full refund if deadlines are missed. The service provider leases computing resources from an infrastructure provider in a two-tier scheme: long-term leasing of basic infrastructure, consisting of heterogeneous parallel service nodes, each modeled as a multi-server queue, and short-term leasing of external servers. Given the intractability of computing an optima…
Policy iteration techniques for multiple-server dispatching rely on the computation of value functions. In this context, we consider the continuous-space M/G/1-FCFS queue endowed with an arbitrarily-designed cost function for the waiting times of the incoming jobs. The associated relative value function is a solution of Poisson's equation for Markov chains, which in this work we solve in the Laplace transform domain by considering an ancillary, underlying stochastic process extended to (imaginary) negative backlog states. This construction enables us to issue closed-form relative value functio…
In this paper we study the uniform stability properties of two classes of parallel server networks with multiple classes of jobs and multiple server pools of a tree topology. These include a class of networks with a single non-leaf server pool, such as the 'N' and 'M' models, and networks of any tree topology with class-dependent service rates. We show that with $\sqrt{n}$ safety staffing, and no abandonment, in the Halfin--Whitt regime, the diffusion-scaled controlled queueing processes are exponentially ergodic and their invariant probability distributions are tight, uniformly over all stati…
