Throughput and Delay Optimality of Power-of-d Choices in Inhomogeneous Load Balancing Systems

Load Balancing in Heterogeneous Server Clusters: Insights From a Product-Form Queueing Model

Efficiently exploiting servers in data centers requires performance analysis methods that account not only for the stochastic nature of demand but also for server heterogeneity. Although several recent works proved optimality results for heterogeneity-aware variants of classical load-balancing algorithms in the many-server regime, we still lack a fundamental understanding of the impact of heterogeneity on performance in finite-size systems. In this paper, we consider a load-balancing algorithm that leads to a product-form queueing model and can therefore be analyzed exactly even when the numbe…

Scalable Load Balancing in Networked Systems: Universality Properties and Stochastic Coupling Methods

We present an overview of scalable load balancing algorithms which provide favorable delay performance in large-scale systems, and yet only require minimal implementation overhead. Aimed at a broad audience, the paper starts with an introduction to the basic load balancing scenario, consisting of a single dispatcher where tasks arrive that must immediately be forwarded to one of $N$ single-server queues. A popular class of load balancing algorithms are so-called power-of-$d$ or JSQ($d$) policies, where an incoming task is assigned to a server with the shortest queue among $d$ servers selected …

Optimal Load Balancing in Bipartite Graphs

Applications in cloud platforms motivate the study of efficient load balancing under job-server constraints and server heterogeneity. In this paper, we study load balancing on a bipartite graph where left nodes correspond to job types and right nodes correspond to servers, with each edge indicating that a job type can be served by a server. Thus edges represent locality constraints, i.e., each job can only be served at servers which contained certain data and/or machine learning (ML) models. Servers in this system can have heterogeneous service rates. In this setting, we investigate the perfor…

Universality of Power-of-$d$ Load Balancing in Many-Server Systems

We consider a system of $N$ parallel single-server queues with unit exponential service rates and a single dispatcher where tasks arrive as a Poisson process of rate $λ(N)$. When a task arrives, the dispatcher assigns it to a server with the shortest queue among $d(N)$ randomly selected servers ($1 \leq d(N) \leq N$). This load balancing strategy is referred to as a JSQ($d(N)$) scheme, marking that it subsumes the celebrated Join-the-Shortest Queue (JSQ) policy as a crucial special case for $d(N) = N$. We construct a stochastic coupling to bound the difference in the queue length processes bet…

异步负载均衡与自动扩缩容：平均场极限与最优设计

Asynchronous Load Balancing and Auto-scaling: Mean-Field Limit and Optimal Design

本文提出结合经典算法与自动扩缩容的马尔可夫负载均衡框架，首次研究异步运行场景，找到实现延迟与相对能量最优的扩缩容规则条件，并提出优于同步方案的规则族。

异构系统Halfin-Whitt机制下速度感知最短队列策略的渐近最优性

Asymptotic Optimality of the Speed-Aware Join-the-Shortest-Queue in the Halfin-Whitt Regime for Heterogeneous Systems

本文证明速度感知最短队列（SA-JSQ）策略在Halfin-Whitt流量机制下对异构服务器系统具有渐近最优性，通过马尔可夫过程弱收敛到二维反射OU过程、Stein方法及耦合论证，证明其在稳态下渐近最小化扩散尺度化的作业总数和等待作业数。

Scalable load balancing in networked systems: A survey of recent advances

The basic load balancing scenario involves a single dispatcher where tasks arrive that must immediately be forwarded to one of $N$ single-server queues. We discuss recent advances on scalable load balancing schemes which provide favorable delay performance when $N$ grows large, and yet only require minimal implementation overhead. Join-the-Shortest-Queue (JSQ) yields vanishing delays as $N$ grows large, as in a centralized queueing arrangement, but involves a prohibitive communication burden. In contrast, power-of-$d$ or JSQ($d$) schemes that assign an incoming task to a server with the shorte…

Asymptotic Optimality of Speed-Aware JSQ for Heterogeneous Systems

The Join-the-Shortest-Queue (JSQ) load-balancing scheme is known to minimise the average delay of jobs in homogeneous systems consisting of identical servers. However, it performs poorly in heterogeneous systems where servers have different processing rates. Finding a delay optimal scheme remains an open problem for heterogeneous systems. In this paper, we consider a speed-aware version of the JSQ scheme for heterogeneous systems and show that it achieves delay optimality in the fluid limit. One of the key issues in establishing this optimality result for heterogeneous systems is to show that …

具有个体异构服务器的负载均衡

Load Balancing with Individually Heterogeneous Servers

研究具有个体异构服务器的系统中JSQ负载均衡策略的扩散极限，基于测度值过程开发框架统一分析，确定区分不同平局决胜规则性能的关键对象，找到渐近最优的平局决胜规则并通过耦合比较证明结果。

带记忆的$d$选择幂算法：流体极限与最优性

Power-of-$d$-Choices with Memory: Fluid Limit and Optimality

本文提出带本地记忆的$d$选择幂负载均衡算法，证明其在系统负载满足条件时具有渐近最优性，不满足时队列长度有严格上界，量化了记忆对随机负载均衡的性能提升作用。

异构服务系统中跳过最长队列的重交通最优性

Heavy-traffic Optimality of Skip-the-Longest-Queues in Heterogeneous Service Systems

研究异构并行服务系统中的超低通信开销负载均衡策略$k$-SLQ-$d$，证明其在满足特定条件时具有吞吐量最优性，并在重交通下渐近延迟最优。

Asymptotically Optimal Load Balancing Topologies

We consider a system of $N$ servers inter-connected by some underlying graph topology $G_N$. Tasks arrive at the various servers as independent Poisson processes of rate $λ$. Each incoming task is irrevocably assigned to whichever server has the smallest number of tasks among the one where it appears and its neighbors in $G_N$. Tasks have unit-mean exponential service times and leave the system upon service completion. The above model has been extensively investigated in the case $G_N$ is a clique. Since the servers are exchangeable in that case, the queue length process is quite tractable, an…