ModiPick：面向移动深度推理的SLA感知精度优化
ModiPick: SLA-aware Accuracy Optimization For Mobile Deep Inference
摘要：
移动应用越来越多地利用复杂的深度学习模型来提供功能，例如，图像识别，其需要高预测精度。这样的模型可能是计算和内存密集型的，即使是对于较新的移动设备，因此通常托管在功能强大的远程服务器上。然而，目前基于云的推理服务采用静态模型选择方法，这可能是次优的满足应用SLA（服务水平协议），因为它们不能考虑固有的动态移动环境。我们介绍了一种基于云的技术，称为ModiPick，动态选择最合适的模型，为每个推理请求，并适应其选择，以匹配不同的SLA和执行时间预算所造成的可变的移动环境。ModiPick的核心思想是在运行时使用托管深度学习模型池进行推理速度和准确性的权衡。因此，ModiPick屏蔽了不可预测的推理时间预算，因此满足SLA目标，同时提高了移动网络约束内的准确性。我们通过基于原型系统的实验和模拟来评估ModiPick。我们表明，ModiPick实现了与贪婪方法相当的推理准确性，同时将SLA依从性提高了88.5%。
英文摘要：
Mobile applications are increasingly leveraging complex deep learning models to deliver features, e.g., image recognition, that require high prediction accuracy. Such models can be both computation and memory-intensive, even for newer mobile devices, and are therefore commonly hosted in powerful remote servers. However, current cloud-based inference services employ static model selection approach that can be suboptimal for satisfying application SLAs (service level agreements), as they fail to account for inherent dynamic mobile environment. We introduce a cloud-based technique called ModiPick that dynamically selects the most appropriate model for each inference request, and adapts its selection to match different SLAs and execution time budgets that are caused by variable mobile environments. The key idea of ModiPick is to make inference speed and accuracy trade-offs at runtime with a pool of managed deep learning models. As such, ModiPick masks unpredictable inference time budgets and therefore meets SLA targets, while improving accuracy within mobile network constraints. We evaluate ModiPick through experiments based on prototype systems and through simulations. We show that ModiPick achieves comparable inference accuracy to a greedy approach while improving SLA adherence by up to 88.5%.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
