arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2510.02613cs.DC
收藏

ElasticMoE:一种面向混合专家模型的高效自动扩缩容方法

ElasticMoE: An Efficient Auto Scaling Method for Mixture-of-Experts Models

Huawei Technologies Canada(华为加拿大技术有限公司);Huawei Technologies China(华为中国技术有限公司)

Gursimran Singh, Timothy Yu, Haley Li, Cheng Chen, Hanieh Sadri, Qintao Zhang, Yu Zhang, Ying Xiong, Yong Zhang, Zhenan Fan

AI总结:

针对MoE大语言模型现有扩缩容方法粒度粗、延迟高、易停机的问题,提出ElasticMoE框架,通过解耦推理与内存、零拷贝重映射、虚拟内存专家重分配实现细粒度零停机扩缩容,实验验证其在延迟、吞吐量、SLO达成率上均有显著提升。

摘要:

混合专家(Mixture-of-Experts, MoE)模型通过每个token仅激活一小部分专家,有望实现大语言模型(large language models, LLMs)的高效扩展,但其并行推理流水线给弹性服务带来了挑战。现有策略存在不足:水平扩缩容需要配置当前架构的完整副本,通常需要数十到数百个加速器,导致粒度粗、配置延迟长、过度配置成本高。垂直扩缩容可实现更精细的调整,但通常需要重启实例,会造成停机。这些局限性使得现有方法无法适配云部署中常见的突发、短生命周期流量模式。\n我们提出了ElasticMoE,这是一种面向MoE大语言模型的弹性扩缩容框架,可实现细粒度、低延迟、零停机的扩缩容。ElasticMoE将推理执行与内存操作解耦,使扩缩容步骤可与服务同时进行。HBM管理模块(HBM Management Module, HMM)通过零拷贝重映射复用权重和KV缓存,同时高带宽点对点传输让新增加速器上线时不中断服务。基于虚拟内存的专家重分配机制无需昂贵的缓冲区重分配即可迁移MoE专家,降低了专家并行重配置期间的峰值内存使用量。\n我们在昇腾(Ascend)NPU上使用三种主流MoE大语言模型进行评估,结果表明,与基线方法相比,ElasticMoE的扩容延迟最高降低9倍,扩容期间的吞吐量最高提升2倍,且显著改善了服务等级目标(SLO)达成率。通过实现细粒度、并发式且干扰极小的扩缩容,ElasticMoE提升了在动态云环境中部署大规模MoE大语言模型的实用性。

英文摘要:

Mixture-of-Experts (MoE) models promise efficient scaling of large language models (LLMs) by activating only a small subset of experts per token, but their parallelized inference pipelines make elastic serving challenging. Existing strategies fall short: horizontal scaling provisions entire replicas of the current configuration, often tens to hundreds of accelerators, leading to coarse granularity, long provisioning delays, and costly overprovisioning. Vertical scaling offers finer adjustments but typically requires instance restarts, incurring downtime. These limitations make current approaches ill-suited for the bursty, short-lived traffic patterns common in cloud deployments. We present ElasticMoE, an elastic scaling framework for MoE LLMs that achieves fine-grained, low-latency, and zero-downtime scaling. ElasticMoE decouples inference execution from memory operations, enabling scaling steps to proceed concurrently with serving. An HBM Management Module (HMM) reuses weights and KV caches via zero-copy remapping, while high-bandwidth peer-to-peer transfers bring newly added accelerators online without interrupting service. A virtual memory based expert redistribution mechanism migrates MoE experts without costly buffer reallocations, reducing peak memory usage during expert parallelism reconfiguration. Our evaluation on Ascend NPUs with three popular MoE LLMs shows that ElasticMoE achieves up to 9x lower scale-up latency, up to 2x better throughput during scaling, and significantly improves SLO attainment compared to baselines. By enabling fine-grained, concurrent scaling with minimal disruption, ElasticMoE advances the practicality of deploying massive MoE LLMs in dynamic cloud environments.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑