MOPAR：面向无服务器平台上深度学习推理服务的模型划分框架
MOPAR: A Model Partitioning Framework for Deep Learning Inference Services on Serverless Platforms
AI总结：
针对无服务器平台上深度学习推理服务资源利用不均的问题，本文提出MOPAR模型划分框架，通过垂直切片、子切片并行优化及数据压缩与共享内存技术，在12个模型上实现资源效率平均提升27.62%、延迟降低约5.52%并显著节约成本。
摘要：
凭借弹性能力和按使用付费的成本模式，在无服务器平台上部署深度学习推理服务（DLISs）正成为一种普遍趋势。然而，当DLISs作为单一函数部署在无服务器平台上时，深度学习模型中不同层对资源需求的差异会阻碍资源利用并增加成本。为了解决这一问题，我们提出了一个名为MOPAR的模型划分框架。该工作基于DLISs的两种资源使用模式：由于资源主导（RD）算子和层堆叠的存在而产生的全局差异和局部相似性。考虑到这些模式，MOPAR采用一种混合方法，首先将深度学习模型垂直划分为多个由相似层组成的切片，以提高资源效率。包含RD算子的切片被进一步划分为多个子切片，从而支持并行优化以降低推理延迟。此外，MOPAR综合运用数据压缩和共享内存技术，以抵消切片间通信带来的额外时间。我们实现了MOPAR的原型，并在OpenFaaS和AWS Lambda上使用四类共12个深度学习模型评估其有效性。实验结果表明，MOPAR平均可将DLISs的资源效率提高27.62%，同时将延迟降低约5.52%。此外，基于Lambda的定价，使用MOPAR运行DLISs的成本降低了约2.58倍。
英文摘要：
With its elastic power and a pay-as-you-go cost model, the deployment of deep learning inference services (DLISs) on serverless platforms is emerging as a prevalent trend. However, the varying resource requirements of different layers in DL models hinder resource utilization and increase costs, when DLISs are deployed as a single function on serverless platforms. To tackle this problem, we propose a model partitioning framework called MOPAR. This work is based on the two resource usage patterns of DLISs: global differences and local similarity, due to the presence of resource dominant (RD) operators and layer stacking. Considering these patterns, MOPAR adopts a hybrid approach that initially divides the DL model vertically into multiple slices composed of similar layers to improve resource efficiency. Slices containing RD operators are further partitioned into multiple sub-slices, enabling parallel optimization to reduce inference latency. Moreover, MOPAR comprehensively employs data compression and share-memory techniques to offset the additional time introduced by communication between slices. We implement a prototype of MOPAR and evaluate its efficacy using four categories of 12 DL models on OpenFaaS and AWS Lambda. The experiment results show that MOPAR can improve the resource efficiency of DLISs by 27.62\% on average, while reducing latency by about 5.52\%. Furthermore, based on Lambda's pricing, the cost of running DLISs is reduced by about 2.58 $\times$ using MOPAR.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
