AlpaServe:面向深度学习服务的带模型并行的统计复用
AlpaServe: Statistical Multiplexing with Model Parallelism for Deep Learning Serving
AI总结:
该研究提出AlpaServe系统,利用模型并行实现多设备统计复用,在多模型服务场景下平衡并行开销与延迟优化,可大幅提升请求处理速率与突发承载能力。
摘要:
模型并行通常被视为一种将单个大型深度学习模型扩展至超出单设备内存限制的方法。本文中,我们证明在为多个模型提供服务时,模型并行还可用于多设备的统计复用,即便单个模型能够装入单设备内存。本研究揭示了模型并行引入的开销,与突发工作负载下利用统计复用降低服务延迟的机会之间的基本权衡关系。我们探索了这一新的权衡空间,并提出了一种新型服务系统AlpaServe,它能确定在分布式集群上放置并并行化大型深度学习模型集合的高效策略。基于生产工作负载的评估结果表明,AlpaServe能以最高10倍的速率处理请求,或承受6倍的突发程度,同时可让超过99%的请求满足延迟约束。
英文摘要:
Model parallelism is conventionally viewed as a method to scale a single large deep learning model beyond the memory limits of a single device. In this paper, we demonstrate that model parallelism can be additionally used for the statistical multiplexing of multiple devices when serving multiple models, even when a single model can fit into a single device. Our work reveals a fundamental trade-off between the overhead introduced by model parallelism and the opportunity to exploit statistical multiplexing to reduce serving latency in the presence of bursty workloads. We explore the new trade-off space and present a novel serving system, AlpaServe, that determines an efficient strategy for placing and parallelizing collections of large deep learning models across a distributed cluster. Evaluation results on production workloads show that AlpaServe can process requests at up to 10x higher rates or 6x more burstiness while staying within latency constraints for more than 99% of requests.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。