MDInference：平衡移动应用的推理准确性和延迟
MDInference: Balancing Inference Accuracy and Latency for Mobile Applications
摘要：
深度神经网络允许移动设备将广泛的功能整合到用户应用程序中。然而，这些模型的计算复杂性使得难以在资源受限的移动设备上有效地运行它们。先前的工作通过降低模型复杂性或利用强大的云服务器来解决在移动应用程序中支持深度学习的问题。这些方法都只关注移动推理的一个方面，因此它们通常会牺牲整体性能。在这项工作中，我们介绍了一种整体的方法来设计移动深度推理框架。我们首先确定移动深度推理的准确性和延迟的关键目标，以及实现这些目标必须满足的条件。我们展示了我们的整体方法，通过设计一个假设的框架称为MDInference。该框架利用了两种互补的技术;模型选择算法，从一组基于云的深度学习模型中进行选择，以提高推理准确性;以及设备上的请求复制机制，以限制延迟。通过仿真驱动的模拟，我们表明，MDInference提高了超过40%的静态方法的聚合准确性，而不会导致SLA违规。此外，我们还表明，在目标延迟为250 ms的情况下，MDInference在更快的大学网络上提高了99.74%的总体准确率，在住宅网络上提高了96.84%的总体准确率。
英文摘要：
Deep Neural Networks are allowing mobile devices to incorporate a wide range of features into user applications. However, the computational complexity of these models makes it difficult to run them effectively on resource-constrained mobile devices. Prior work approached the problem of supporting deep learning in mobile applications by either decreasing model complexity or utilizing powerful cloud servers. These approaches each only focus on a single aspect of mobile inference and thus they often sacrifice overall performance. In this work we introduce a holistic approach to designing mobile deep inference frameworks. We first identify the key goals of accuracy and latency for mobile deep inference and the conditions that must be met to achieve them. We demonstrate our holistic approach through the design of a hypothetical framework called MDInference. This framework leverages two complementary techniques; a model selection algorithm that chooses from a set of cloud-based deep learning models to improve inference accuracy and an on-device request duplication mechanism to bound latency. Through empirically-driven simulations we show that MDInference improves aggregate accuracy over static approaches by over 40% without incurring SLA violations. Additionally, we show that with a target latency of 250ms, MDInference increased the aggregate accuracy in 99.74% cases on faster university networks and 96.84% cases on residential networks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
