ReviveMoE: 大规模MoE大语言模型推理部署中硬件故障的快速恢复
ReviveMoE: Fast Recovery for Hardware Failures in Large-Scale MoE LLM Inference Deployments
AI总结：
ReviveMoE通过无需重启服务实例实现大规模MoE大语言模型部署中的快速故障恢复，基于华为的xDeepServe平台和XCCL通信库。
摘要：
随着LLM部署的硬件规模不断扩大，系统中单个故障的概率显著增加，云运营商必须考虑鲁棒的应对措施来处理这些不可避免的故障。一种常见的恢复方法是简单地重新启动LLM服务实例；然而，在模型即服务（MaaS）推理设置中，这种做法成本较高，因为重新加载模型权重和重新编译计算图会为 incoming 请求引入显著的延迟。我们提出了ReviveMoE，一种无需重新启动服务实例即可在大规模LLM部署中实现快速故障恢复的方法。ReviveMoE旨在支持传统的LLM架构，该架构将MoE和注意力放在同一硬件上，以及解耦架构，将MoE与注意力分开。ReviveMoE集成到华为云的MaaS中，基于华为的xDeepServe服务平台和XCCL通信库。
英文摘要：
As LLM deployments scale over more hardware, the probability of a single failure in a system increases significantly, and cloud operators must consider robust countermeasures to handle these inevitable failures. A common recovery approach is to simply restart the LLM serving instance; however, this is costly in model-as-a-service (MaaS) inference settings, where reloading model weights and recompiling computation graphs can introduce significant delays to incoming requests. We propose ReviveMoE, a method for rapid failure recovery in large-scale LLM deployments without restarting the serving instance. ReviveMoE is designed to support both the traditional LLM architecture, which collocates MoE and attention on the same hardware, and the disaggregated architectures, which separate MoE from attention. Integrated into Huawei Cloud's MaaS, ReviveMoE is built on top of Huawei's xDeepServe serving platform and the XCCL communications library.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
