探索虚拟化对深度学习应用程序可用性的影响
Exploring the Impact of Virtualization on the Usability of the Deep Learning Applications
摘要：
基于深度学习（DL）的应用正变得越来越流行，并以前所未有的速度发展。虽然许多研究工作正在进行，以增强深度神经网络（DNN）-DL应用程序的核心-这些应用程序在云和边缘系统中的实际部署挑战，以及它们对应用程序可用性的影响尚未得到充分研究。特别是，部署由云和边缘提供的不同虚拟化平台对DL应用程序的可用性（在端到端（E2 E）推理时间方面）的影响仍然是一个悬而未决的问题。重要的是，资源弹性（通过扩展），CPU固定和处理器类型（CPU与GPU）配置已被证明对虚拟化开销有影响。因此，本研究的目标是研究这些潜在的决定性部署选项对E2 E性能的影响，因此，DL应用程序的可用性。为此，我们衡量四种流行的执行平台（即，裸机，虚拟机（VM），容器，容器中的VM）的E2 E推理时间的四种类型的DL应用程序的影响，改变处理器配置（扩展，CPU固定）和处理器类型。这项研究揭示了一组有趣的，有时甚至是违反直觉的发现，这些发现可以作为云解决方案架构师在各种系统中有效部署DL应用程序的最佳实践。值得注意的发现是，解决方案架构师必须了解DL应用程序的特性，特别是其预处理和后处理要求，以便能够最佳地选择和配置执行平台，确定GPU的使用，并决定有效的扩展范围。
英文摘要：
Deep Learning-based (DL) applications are becoming increasingly popular and advancing at an unprecedented pace. While many research works are being undertaken to enhance Deep Neural Networks (DNN) -- the centerpiece of DL applications -- practical deployment challenges of these applications in the Cloud and Edge systems, and their impact on the usability of the applications have not been sufficiently investigated. In particular, the impact of deploying different virtualization platforms, offered by the Cloud and Edge, on the usability of DL applications (in terms of the End-to-End (E2E) inference time) has remained an open question. Importantly, resource elasticity (by means of scale-up), CPU pinning, and processor type (CPU vs GPU) configurations have shown to be influential on the virtualization overhead. Accordingly, the goal of this research is to study the impact of these potentially decisive deployment options on the E2E performance, thus, usability of the DL applications. To that end, we measure the impact of four popular execution platforms (namely, bare-metal, virtual machine (VM), container, and container in VM) on the E2E inference time of four types of DL applications, upon changing processor configuration (scale-up, CPU pinning) and processor types. This study reveals a set of interesting and sometimes counter-intuitive findings that can be used as best practices by Cloud solution architects to efficiently deploy DL applications in various systems. The notable finding is that the solution architects must be aware of the DL application characteristics, particularly, their pre- and post-processing requirements, to be able to optimally choose and configure an execution platform, determine the use of GPU, and decide the efficient scale-up range.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
