ENOVA：面向成本效益与稳定性的无服务器LLM服务自动扩缩容
ENOVA: Autoscaling towards Cost-effective and Stable Serverless LLM Serving
AI总结：
针对多GPU集群上无服务器LLM服务因应用混部导致的服务质量与GPU利用率低的问题，提出ENOVA服务，通过解构执行流程设计配置推荐、性能检测模块及部署执行引擎，实现更优的自动扩缩容效果，适用于大型在线系统。
摘要：
随着大语言模型（LLM）后端系统日益普及，在多GPU集群上部署具备自动扩缩容能力的稳定无服务器LLM服务已成普遍且必要的需求。但多GPU集群中应用的多样性与混部会导致服务质量低下、GPU利用率不足，存在诸多挑战。为解决这些问题，我们构建了ENOVA——一款面向无服务器LLM服务的部署、监控与自动扩缩容服务。ENOVA全面解构了LLM服务的执行流程，基于此设计了可在任意GPU集群上自动部署的配置推荐模块，以及用于自动扩缩容的性能检测模块。在此之上，ENOVA实现了用于多GPU集群调度的部署执行引擎。实验结果表明，ENOVA的性能显著优于其他现有最优方法，适合在大型在线系统中广泛部署。
英文摘要：
Since the increasing popularity of large language model (LLM) backend systems, it is common and necessary to deploy stable serverless serving of LLM on multi-GPU clusters with autoscaling. However, there exist challenges because the diversity and co-location of applications in multi-GPU clusters will lead to low service quality and GPU utilization. To address them, we build ENOVA, a deployment, monitoring and autoscaling service towards serverless LLM serving. ENOVA deconstructs the execution process of LLM service comprehensively, based on which ENOVA designs a configuration recommendation module for automatic deployment on any GPU clusters and a performance detection module for autoscaling. On top of them, ENOVA implements a deployment execution engine for multi-GPU cluster scheduling. The experiment results show that ENOVA significantly outperforms other state-of-the-art methods and is suitable for wide deployment in large online systems.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
